In medical terms the condition you are describing is called genu valgus. This knee deformity causes a transfer of weight to the inner (medial) side of the joint when it would otherwise be spread more evenly across the joint.
Arthritis on just one side of the joint can occur. This is called unicompartmental arthritis. Osteotomy is frequently done for this problem. An osteotomy attempts to transfer the forces of the body weight onto the normal, non-arthritic side, by actually changing the angle of the bone.
This is done by cutting a wedge-shaped piece out of the bone and inserting it on the collapsed side. This is called an opening wedge osteotomy. Or it can be done by cutting out a wedge and collapsing the gap on the other side (closing wedge) osteotomy. Opening wedge is the more commonly done operation.
If this operation can't be done for some reason, then unicompartmental knee replacement (UKR) is another option. The damaged half of the joint is removed and replaced by a single-sided joint replacement. For genu valgus, just the medial side of the joint is replaced.
The first step is to see an orthopedic surgeon for an exam. X-rays will likely be taken and then the best treatment plan can be determined.