It's possible but you'll need someone to look at it for you. Make a follow-up appointment with whoever made or fitted the orthotics for you. You may be slightly over-corrected or something may have worn down on the orthotic to change your foot and ankle position from where it was to start.

It's also possible the shoes need to be replaced. Don't make any changes until your situation can be re-checked. Once the foot is in good alignment then the kneecap can track properly and knee pain goes away. New symptoms can crop up if there is a separate problem anywhere from toes to hip. A follow-up or second evaluation may be needed to check this out.