Here's an experiment for you. Take a look at the following chord chart.
Does it look inviting and pleasant to improvise over?
Probably not, and that's because you are evaluating it through your eyes and analytical brain. The message most likely is THIS LOOKS HARD.
If you were to improvise over these changes, what notes would your eyes and calculating brain tell you to play? And while you're figuring all that out, is music being produced by your instrument?
Now, play the following rhythm track created for this exercise. A synth tone will play the first two phrases, find those starting notes on your instrument–maybe sing them–then listen for the count off and play the melody. The tune is the standard, Tangerine.
Do not look at the chord chart and do not calculate the key. The only sense you can use is your ear.
Playing the melody at first may be hard. Sing the melody. Listen to the pitches of the notes as you sing. Now, play it on your instrument.
If you can play the melody with flow and style, next improvise over the track. I've given you three choruses of the form over which you can play the head or improvise.
While you play over the changes, turn off that part of your brain that is wondering what the chords are and what scales to play. STOP THAT.
Just listen. Listen deep into the track. Can you hear the bass? Can you hear the piano. Listen.
You don't have to sound like a jazz genius over the changes, but I'm betting that you sound a heck of a lot better than if you improvised over the form of the tune staring at those chord changes.
Give your ear a chance. It hears more than you give it credit for.
Continue to practice your technique, run your scales, and learn about harmony, but when the time comes to play music over changes, just listen.
How did this experiment work for you?