Based on the specific focal mechanism, magnitude, fault distance, and site condition, 69 ground motion records are selected and grouped. Nspectra software is used to calculate the inelastic displacement response spectrum of a single isolated structure under ground motion. The influence of fault distance, field condition, magnitude, and damping ratio on the inelastic displacement spectrum is analyzed, and the influence of the mechanical parameters of seismic isolation layer on the seismic energy dissipation is discussed. The results indicate that compared with the far field, the base-isolated structure in the near field is more unfavorable. Moreover, the seismic isolation layer displacement spectrum is greatly influenced by field conditions, earthquake acceleration, and speed. With the increase of the fault distance, the attenuation of displacement spectrum values increases, and the displacement spectrum of the base isolated layer in the field of soft soil is more than that of hard soil. The influence of earthquake magnitude on the spectrum shape is not obvious; however, it can cause a scaling effected on the displacement spectrum of the base isolated layer. When the damping ratio is less than 0.4, the value of displacement spectrum of the seismic isolation layer differs largely with different ground motion characteristics; when the damping ratio is greater than 0.4, the value of displacement spectrum basically become the same. The energy dissipation of the seismic isolation layer with a small yielding force (restoring force/gravity less than or equal to 1) increases with the increase of natural period.