One more point though I would hesitate to share if it's more likely to confuse: Even though all computer uses binary, how the CPU use binary is not universal, kind of akin to how Latin alphabet can be used among several languages but knowing the alpahbet does not enable you to understand the other languages. Every CPU (family) has their own instruction set (e.g. their native language) that may be similar but not identical to others' instruction set. Writing high-level language allow us to share the same function among several possible instruction sets. So yes, you could write in binary directly but that would mean you are now tied to a specific CPU. There's no guarantee that even the next generation CPU from same manufacturer will use the same instruction set, and that would seriously crimp on our productivity.
But this may end up confusing some people more, though.
