Optimal Codebook Generation And Adaptation In Compression, Communications And Machine Learning