Many of the issues worth taking into account are the same when writing software which is designed to be helpful for access as when trying to follow good design.
If your software is only usable through a graphical interface then it can be very hard to make it usable for someone who can't see. If it's only usable through a line oriented interface, then someone who can't type will have difficulties.
Provide keyboard shortcuts as well as the use of the normal X pointer (generally the mouse). You can almost certainly rely on the user being able to generate key presses to your application.
If it's easy to change fonts then people will be able to change to one they can read. If the colour scheme can be changed then people who are colour blind will be more likely to be able to use it. If fonts can be changed easily then the visually impaired will find your software more useful.
If you have a number of people use your software, each with different access problems then they will be more likely to point up specific problems. Obviously, this won't be practical for everybody, but you can always ask for feedback.
Where possible, make it clear what different parts of your program are what. Format error messages in a specific way to identify them. Under X, make sure each pane of your window has a name so that any screen reader software can identify it.