A lot of people with low back pain share your concerns. Often, once people see that they can safely do activities without hurting themselves, they feel more comfortable doing the things they used to do.
This effect was observed in a recent study. Patients who did exercise therapy (physical therapy, muscle training, or low-impact aerobics) found that they were better able to go back to their regular activities. They felt more in control of their pain and less worried that work and physical activity would cause more pain.
You may want to work with a doctor or physical therapist to figure out a program that will help you be active in a safe, supported way. A program that involves some kind of physical activity will speed your recovery by getting you moving.