The plus sign is more difficult than a letter for a number since the correct definition of plus sign did not come until the 19th century with Grassmann, Dedekind and Peano. But letters for numbers were used in prior centuries. Even today, many can use a letter for a number, but almost no one can define the plus sign even close to the correct definitions.
Right Additive Identities
x+0 = x
x+y’ = (x+y)’
Here ‘ means next number, successor. So 1 = 0′, 2 = 1’.
This definition of addition is know to very few. But using letters for numbers is now required to graduate high school in the US.
This implies we can use letters for numbers as soon as we use the plus sign, so in grade K or grade 1.
As soon as we start using the symbols 0,1, 2 we should use letters for numbers and use the prime notation of 1=0′, etc.
If you compare what children learn to speak in terms of grammar, it is obvious this is no worse.
If children can learn to spell different words with the same letters, then can learn to represent different numbers with a letter.