Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications or problems in scientific machine learning. In this talk, we will introduce the sparse random feature expansion to obtain parsimonious random feature models. Specifically, we leverage ideas from compressive sensing to generate random feature expansions with theoretical guarantees even in the data-scarce setting. We also present a random feature model for approximating high-dimensional sparse additive functions and a sparse random mode decomposition to extract intrinsic modes from challenging time-series data. Comparisons show that our proposed approaches perform better or are comparable to other state-of-the-art or popular methods. Applications of our methods on identifying important variables in high-dimensional settings as well as on decomposing music pieces and visualizing black-hole mergers will be addressed.