Objective: In general, financial time series such as stock indexes have nonlinear, mutable and noisy behavior. Structural and statistical models and machine learning-based models are often unable to accurately predict series with such a behavior. Accordingly, the aim of the present study is to present a new hybrid model using the advantages of the GMDH method and Non-dominated Sorting Genetic Algorithm II (NSGA II) to, more accurately, predict the trend of movement and volatility of Tehran Stock Exchange Price Index, and to compare its ability with the ARIMA model based on RMSE, MAPE, and TIC error assessment criterions. Methods: For this purpose, the data of Tehran Stock Exchange Dividend and Price Index (TEDPIX) was used during the period of October 2008 to September 2013. The hybrid model NSGA II-GMDH utilizes the GMDH network as a model resistant to nonstationary and noisy data for prediction and uses the NSGA II multi-objective optimization algorithm to minimize predictive error and select the optimal input variables. Results: The results of the proposed hybrid model in this study indicated a lower error and more prediction accuracy compared to ARIMA model for out-of-sample data based on all three error criterions. Conclusion: The empirical findings of the study showed that the proposed model has higher flexibility and capability in covering unstable changes in the total index movement trend.