Machine learning, like most technology, has the capacity to surpass human capabilities in many ways. However, there are some human skills that machine learning is ineffective at imitating. This paper examines jazz improvisation in this context, specifically focusing on the musical knowledge that must be embedded into software for machine learning on musical data to take place, and how much of this knowledge cannot be represented to a neural network without massive advancements in technology.