I largely agree with fire7's response but to add to it I think it's important to realise that what people think of as genius is often honed to specializations even within the class they are considered a genius.
Even at the peak level of sport, a tennis star may be considered the best 'on grass' player, or the best 'doubles' player. The chess analogy has this with the various playstyles, blitz, classical, rapid etc. with different components to learn for each and even some elite players specialising in one over the other.
Programming is an immensely broad topic where even knowing the rules may not make you proficient for a particular use case. I used to work with a few seasoned C greybeards years ago (who contributed to VLC codecs from memory) who were unquestionably leagues ahead of me in their programming knowledge, but I'd be willing to bet they'd struggle to wrap their head around rapid development for data science projects or quick game prototypes, simply because their style/design ethos is about building long-term reliable/efficient systems using older waterfall methodologies. This doesn't mean they can't relearn, but ultimately there are only so many hours in the day and pivoting to a new specialisation will naturally result in lost knowledge in the previous or conflicts in styles.
I think a large part of your upbringing is about forming your 'logic of learning' if you will. I had an interesting debate once with an artist friend of mine who described creativity in art as a form of logic. To her the steps you take to compose a piece of art IS logical and follows a pattern, but the steps she described to me were foreign and in some cases counter-intuitive to what I considered logical. You ARE a blank slate as a child, but there's only so much you can learn in a given amount of time and when 2 forms of logic/learning are conflicting, I feel it creates a barrier for some and importantly, the people we consider 'geniuses' essentially never changed to begin with and started from the earliest ages.
While I'm sure I could teach this friend of mine how to code, I think they may always inherently struggle to problem solve in code as internally their logic systems are conflicting. Similarly, I think I could be taught the fundamentals of art and composition, but would still struggle to achieve true creativity.
That does not mean you shouldn't try, and everybody can achieve great things in any field through time/effort. But yes, I think your 'potential' in something is limited by the amount of 'somethings' you invest in.
