As a knowledge acquisition theory, constructivism describes information processing mechanisms behind infants' cognitive development. When infants play with the world around them, they exhibit amazing abilities to generate novel behaviors in unseen situations and explore actively to learn the best while lacking extrinsic rewards from the environment. These abilities are critical to achieving autonomous intelligent agents (such as robots). In this article, we seek to understand and replicate some of the abilities in infants' play and propose a computational framework based on the constructivist learning paradigm, which enables sense-making and knowledge construction for self-motivated agents. Furthermore, we evaluate the proposed framework for solving the Small Loop Problem (SLP) and compare its performance with reinforcement-based models. A toolkit of Generating and Analyzing Interaction Traces (GAIT) was introduced to report and explain the fine-grained learning process and the formation of structured behaviors after each decision-making. The result shows that the agent has successfully learned to interact with its environment and avoid unfavorable interactions by using regularities discovered through interaction. Moreover, the proposed framework outperforms reinforcementbased models in learning the goals and adapting behaviors in interacting with dynamic environments.