Defect inversion is a critical quantitative assessment technique to evaluate the integrity of energy transportation systems. In practice, non-stationary states and frequent situation switches in real-world industrial processes make it difficult to acquire ideal sensor signals with precise annotations, there by compromising accurate defect inversion. Simulation systems is a viable solution. However, the dramatic inter-domain disparity between simulated and real system poses challenges to existing studies. Moreover, intra-domain divergence within the real energy transportation systems is also unexplored. To address the above issues, an intra- and inter-domain interactive learning method is proposed for simulation-to-real (sim2real) defect inversion. Specifically, reality-aware feature stylization is proposed which diversifies simulated signals with real style variants, so as to alleviate the domain gap initially. Then, bi-level inter-domain alignment is proposed to not only penalize hard-to-align samples in abstract feature space, but also innovatively exploits intrinsic geometric relations of sensor signals to guide inter-domain gaps. Next, physical-informed intra-domain alignment is proposed where a similarity matrix is constructed to pursue the consistency between sensor signals and physical knowledge, and it is also served as pseudo-labels to constrain inter-domain adaptation, so that inter- and intra- domain adaptation interact with united strength to enhance information exchange. Finally, the proposed method is systematically validated on energy transportation experimental platform and competitive results are obtained, which shows its promise in real-world industrial processes.