A physical concept, the rigid body rule, is applied for the derivation of the higher-order stiffness matrix of a space frame element. The derivation has a physical meaning that is the higher-order stiffness matrix can be derived by regarding there is a set of incremental nodal forces existing on the element, then the element undergoes a small rigid body rotation. The incremental forces should keep their magnitude and follow the rigid body motions. Then taking advantage of the existing geometric stiffness matrix derived by researchers, the higher- order stiffness matrix can be analogy derived without any difficulty. The derived higher-order stiffness matrix has explicit expressions. It can be used at the forces recovery stage in the geometric nonlinear analysis of frame structures. Meanwhile an effective numerical method, the Generalized Displacement Control (GDC) method, was adopted to trace the load-defection curves of the structures. Some numerical examples were tested by taking the proposed higher-order stiffness matrix into consideration in the nonlinear analysis of the structures.