The decision as to how long a cast should be put on a broken arm depends on how the x-ray looks after a set period of time, usually 4 weeks. If your son’s doctor felt that the arm had healed sufficiently and it was safe to take off the cast, this would have been based on what the x-ray showed.
Keeping a cast on for longer won’t necessarily benefit the fracture, however. Even after 6 weeks, the bone can be fragile and you would still have to be careful. It takes a while for the bone to heal completely and regain its full pre-break strength.