Binder is a combination of ingredients which hold the colorful pigment particles together to form the colored pencil's core.
Binder also helps pencil pigment stick to the paper rather than blowing away like pastels.
Binder + Pigment is considered the core. Just the binder is often called the base.
Colored pencil binders typically contain waxes AND oils, plus other ingredients. The exact formulations are proprietary and vary between manufacturers.
The binder plays a major role in a pencil's firmness, glide, adhesion, laydown, and layering potential. It also influences how easily a pencil sheds dust or develops wax bloom.
A pencil's feel and behavior characteristics are basically the sum of the pigments, binders, fillers, and the amount of each ingredient. No single binder ingredient determines whether a pencil is soft, hard, smooth, or easy to layer.
For more on the great Oil vs. Wax debate, read here.