This thesis surveys and creates methods to allow for a mathematically consistent treatmentof non-uniform data sources in machine learning and data compression. These methods are fundamentally rooted in numerical methods for quadrature and interpolation, but are leveraged with appropriate computational tools and techniques to adapt to their respective domains and problem types.