Collaborative filtering is a simple recommendation method which uses related knowledge, but its performance is poor when the data is highly sparse. Factorization machines (FM) can solve the problem of feature combination in the case of data sparsity. Combining with high-order feature extraction via deep neural networks, a series of deep learning prediction models have been proposed and good results have achieved. However, these models mainly depend on the combination of a large number of labels and the understanding of high-order features, whose performance can be seriously degraded when the label categories of data are scarce. In order to solve the problem of recommendation in sparse data and scarce labels, a novel neural network-based recommendation model was proposed which embeds multi-domain labels into a unified domain. First, labels were divided by domain and transformed into feature vectors through embedding layer. Then, the mapping layer was used to embed the feature vectors from the current domain into the unified domain. Finally, the spatial relations of the unified domain-embedded vectors were calculated and predictions were made. Experiments on several open datasets show that the proposed model had higher accuracy and better performance than the mainstream neural network based predicting models. The model overcomes the training bottleneck when label is scarce, and provides a solution for recommender system with limited original data.