With the increasingly stringent requirements on energy saving and environment-friend for Internal Combustion Engine (ICE), new concepts like novel combustion mode and surface texturing technology have emerged. In order to deal with the coupled problem of new combustion mode and surface texture technology, a mixed lubrication theoretical model considering surface textures on cylinder liner inner-wall together with different combustion modes has been built in present work based on a four-cylinder diesel engine. The numerical research was conducted on the change law of the surface tribological performance of textured cylinder liner under different combustion modes. The simulation results indicate that in comparison with the Conventional Combustion (CC) mode, the combustion phase of Premixed Charge Combustion Ignition (PCCI) mode and Retarded Injection (RI) mode vary and their in-cylinder peak pressure are different at the beginning of power stroke due to different injection strategies applied,which lead to significant differences between piston ring radial working loads under these three modes. Near TDC, the mixed lubrication region length of PCCI mode and RI mode increase slightly in comparison with the CC mode and these three boundary lubrication region length are almost the same. However, the thickness of lubricant film between friction pair of the three modes are quite different, resulting in their different tribological properties. Among them, the dimensionless total friction force and the dimensionless friction work of PCCI mode increase with the maximum ratio of 82.37% and 18.41% respectively, while those of RI mode decrease with the maximum ratio of 36.50% and 12.45% respectively, compared with the corresponding parameters of CC mode.