This study aims to develop a second order self-organizing fuzzy neural network (SOFNN) to predict the hourly concentrations of fine particulate matter (PM2.5) for the next 24 h at a regional background station called Shangdianzi (SDZ) in China from 14 to 23 January 2010. The structure of the SOFNN was automatically adjusted according to the sensitivity analysis (SA) of model output and the parameter-learning phase was performed applying a second order gradient (SOG) algorithm. Principal component analysis (PCA) was employed to select the dominating factors for PM2.5 concentrations as the input variables for the SOFNN. It was found that the dominating variables (relative humidity (RH), pressure (Pre), aerosol optical depth (AOD), wind speed (WS) and wind direction (WD)) extracted by PCA agreed well with the characteristics of PM2.5 at SDZ where the PM2.5 concentrations were heavily affected by meteorological parameters and were closely related to AOD. The forecasting results showed that the proposed SOG-SASOFNN performed better than other models with higher coefficient of determination (R2) during both training phase and test phase (0.89 and 0.84, respectively) in predicting PM2.5 concentrations at SDZ. In conclusion, the developed SOG-SASOFNN provided satisfying results for modeling the hourly distribution of PM2.5 at SDZ during the studied period.