In the Property and Casualty (P&C) ratemaking process, it is critical to understand the effect of policyholders’ risk profile to the number and amount of claims, the dependence among various business lines and the claim distributions. To include all the above features, it is essential to develop a regression model which is flexible and theoretically justified. Motivated by the issues above, we propose a class of logit-weighted reduced mixture of experts (LRMoE) models for multivariate claim frequencies or severities distributions. LRMoE is interpretable, as it has two components: Gating functions, which classify policyholders into various latent sub-classes; and Expert functions, which govern the distributional properties of the claims. Also, upon the development of denseness theory in regression setting, we can heuristically interpret the LRMoE as a “fully flexible” model to capture any distributional, dependence and regression structures subject to a denseness condition. Further, the mathematical tractability of the LRMoE is guaranteed since it satisfies various marginalization and moment properties. Finally, we discuss some special choices of expert functions that make the corresponding LRMoE “fully flexible”. In the subsequent paper (Fung et al., 2019b), we will focus on the estimation and application aspects of the LRMoE.