With the rise of models for music generation, a concern arises : “How do we create representations of music that support our models’ generative abilities rather than hinder them?” Powerful models for music generation can only go so far with simple representations of polyphonic music like multi-hot vectors. A good representation of music for humans, would be, functionally, much like words in language syntax; it not only encodes musical events such as when a note is on or off, but it also implies the existence of relationships to other musical structures analogous to words. In this work, we explore the potential of three different methods for creating embeddings of notes sounded together at a given time slice in a piece (this is what we call a chord here, which is broader than the concept of a chord in music theory). We analyze the resulting embeddings with three different strategies in an attempt to elucidate their successes and failures when validated against what is known from music theory. Our results support the hypothesis that a Sequence to Sequence model’s thought vector, obtained with its encoder, can represent major and minor triads such that the relationships between those representations of triads would align with expectations from music theory. We conclude our investigation with suggestions for further rigorous experimentation needed for us to be able to strengthen supporting evidence.