It would be a bad idea to use one-hot encoding where each input word is encoded as 19,999 zero-values and 1 one-value. For these types of NLP problems you should use what is called a word embedding, ...

We will discuss word embeddings this week. Word embeddings represent a fundamental shift in natural language processing (NLP), transforming words into dense vector representations that capture ...