The particle (spherical steel particles) waterjet is a new type of waterjet that can effectively break rocks and other materials. In this project, the simulation combined with the experimental procedure was conducted for optimization of the jet parameters (such as velocity, water pressure, dwell time, concentration, diameter, and standoff distance) for the rock breaking performance modeling of a particle waterjet. In particular, the changes in rock and particle properties following impact were analyzed based on the Smoothed Particle Hydrodynamics (SPH) method by using the Ls-Dyna software. The rock breaking experiments were performed by altering the particle waterjet parameters, with consideration to field applications. Moreover, the effects of particle waterjet parameters on the impact depth and breaking volume were identified. The rock breaking efficiency factor was analyzed by investigating various jet parameters. Overall, the simulation results were in good agreement with the experimental results. The results demonstrated that the rebound effect constituted an important factor in the crater diameter determination. Both the impact depth and breaking volume efficiency factor decreased with the increase in the water inlet velocity. Owing to the effects of deeper craters on the actual impact velocities of the particles, the impact depth and breaking volume efficiency factor decreased as the dwell time increased. Owing to the interference caused by the collisions between the incident and rebound particles, both the impact depth and breaking volume efficiency factor decreased as the concentration increased. The impact depth increased rapidly when the dwell time was below a critical time (40 s). When the particle concentration exceeded a critical value (3%), both the impact depth and breaking volume increased slowly, thus significantly decreasing the rock breaking efficiency. For the appearance of a deeper and wider crater, the optimized standoff distance was ranged from 20 to 30 mm. This analysis provided a realistic approach for the optimization of the particle waterjet parameters in rock breaking.