In this paper, considering the problem of multimodal time series modeling based robot safety surveillance, we present a fast, robust, and versatile measure for executing process identification and anomaly detection through Hierarchical Dirichlet Process Hidden Markov Model (HDPHMM). To effectively improve the robot safety surveillance, first the complex manipulation task into sequences of executing processes was decomposed and then the process identification could be achieved by comparing the log-likelihood value of cumulative observations during robot manipulation. After the process identification, the anomaly detection of each process could also be implemented by discriminating anomalies by the gradient of log-likelihood thresholding from the normal training executions.