Long-Horizon Video Generation with Temporally Consistent Diffusion and Scene-Graph Guidance
DOI:
https://doi.org/10.66857/jirt.v3i2.47Keywords:
Video Generation, Diffusion Models, Scene Graphs, Temporal Consistency, Deep LearningAbstract
The synthesis of high-fidelity, temporally coherent long-horizon videos remains a profound challenge in the domain of generative artificial intelligence. Current diffusion-based approaches often suffer from severe temporal degradation, semantic drift, and structural inconsistency when generating sequences beyond a few seconds. To address these limitations, this paper introduces a novel framework that integrates temporally consistent diffusion models with dynamic scene-graph guidance. By leveraging scene graphs as explicit semantic anchors across frames, the proposed architecture structurally constrains the generative process, ensuring that objects, their attributes, and their interrelationships remain stable over extended durations. The methodology involves a dual-stream architecture where a graph neural network processes sequential scene graphs to condition a cascaded video diffusion model. Furthermore, a specialized spatiotemporal cross-attention mechanism is developed to align latent noise representations with the relational data embedded in the scene graphs. Extensive empirical evaluations on standard video generation benchmarks demonstrate that the proposed method significantly outperforms baseline approaches in both quantitative metrics and qualitative human assessments, particularly in maintaining entity persistence and logical action progression over long time horizons. The findings underscore the critical role of explicit structural representations in overcoming the inherent memory limitations of pure attention-based video generation systems
References
1. Huang, H., Zhang, J., Zhang, J., Xu, J., & Wu, Q. (2020). Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification.IEEE Transactions on Multimedia,23, 1666-1680.
2. Kamal, T., Wenhong, T., Guo, J., Shafiq, M., Shuaihong, J., & Xue, R. (2026). Adaptive Probabilistic Transformer for Medical Image Segmentation. Expert Systems with Applications, 131948.
3. Wang, H., Wei, C., Ren, W., Liu, J., Lin, F., & Chen, W. (2026). Rationalrewards: Reasoning rewards scale visual generation both training and test time.arXiv preprint arXiv:2604.11626.
4. Yang, Y., Bortins, I., Baldwin, D. P., & Zhang, S. (2024). Calibration of dual resolution dual camera structured light systems.Optics and Lasers in Engineering,182, 108472.
5. Wang, J., Malawade, A. V., Zhou, J., Yu, S. Y., & Al Faruque, M. A. (2024, January). Rs2g: Data-driven scene-graph extraction and embedding for robust autonomous perception and scenario understanding. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 7478-7487). IEEE.
6. Jing, G., Gao, P., Hu, Y., Lee, Y., & Zhang, H. (2025, June). ESTI: An Efficient Spatial-Temporal Interaction Network For Video-Based Person Re-Identification. In2025 IEEE International Conference on Multimedia and Expo (ICME)(pp. 1-6). IEEE.
7. Li, B., Zhang, R., Liang, H., Zhang, J., Zhang, J., Chen, X., ... & Wang, J. (2026). Interagent: Physics-based multi-agent command execution via diffusion on interaction graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 15253-15265).
8. Qi Z, Wang S, Su C, et al. Self-regulated learning for egocentric video activity anticipation[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 45(6): 6715-6730.
9. Wang, H., Feng, W., Yu, J., Liu, C., Nie, P., Lin, F., ... & Wei, C. (2026). Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation.arXiv preprint arXiv:2607.05382.
10. Qi, Z., Yuan, Y., Ruan, X., Wang, S., Zhang, W., & Huang, Q. (2024, March). Bias-conflict sample synthesis and adversarial removal debias strategy for temporal sentence grounding in video. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 5, pp. 4533-4541).
11. Zhang B, Wang X, Kao H, et al. Consensus Matrix: A Role Specialized Multi Agent Framework for Structured Collaborative Decision Making in Agentic Visual Media Workflows[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 4613 4622.
12. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of CVPR.
13. Chen, L., Yang, Y., & Zhang, S. (2025). Auto-focusing and auto-exposure method for real-time structured-light 3D imaging.Optical Engineering,64(4), 044105-044105.
14. Zhao, Y., Li, Z., Guo, X., & Lu, Y. (2022). Alignment-guided temporal attention for video action recognition. Advances in Neural Information Processing Systems, 35, 13627-13639.
15. Li, X., Yang, F., Chen, L., & Cai, H. (2016, July). Saliency transfer: An example-based method for salient object detection. In IJCAI (pp. 3411-3417).
16. Qu, W., Wang, J., Gong, Y., Huang, X., & Xiao, L. (2025, June). An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 27325-27335). IEEE.
17. Zhao, Haoyu, et al. "LSTD: Long Short-Term Temporal Diffusion for Video Generation." IEEE Transactions on Multimedia (2026).
18. Xia, Y., & Ma, J. (2025). Grid-guided sparse Laplacian consensus for robust feature matching.IEEE Transactions on Image Processing,34, 1367-1381.
19. Qu, D., & Ma, Y. (2026). Fourier–Transformer Mixer Network for Efficient Video Scene Graph Prediction.Engineering Proceedings,120(1), 16.
20. Gao, P., Lee, Y., Zhang, X., Chen, Z., & Zhang, H. (2026, March). Remember Me: Bridging the Long-Range Gap in LVLMs with Three-Step Inference-Only Decay Resilience Strategies. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 6, pp. 4140-4148).
21. Liu, C., Ma, L., Zhang, X. T., Zhang, Y., Zhang, H., Yang, X., & Tian, F. (2026). Boosting omni-modal language models: Staged post-training with visually debiased evaluation. arXiv preprint arXiv:2605.12034.
22. Zhang, Y., Zhang, Y., Zhao, M., Feng, S., & Cheung, Y. M. (2026). How to Ask the AI: A User Perspective Survey for Large Language Model Prompting.IEEE Transactions on Artificial Intelligence.
23. Jing, G., Gao, P., Lee, Y., Hu, Y., & Zhang, H. (2025). 3D-aided pedestrian representation learning for video-based person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 35 (12), 12830-12845.
24. Xia, Y., Lu, Y., Gao, Y., & Ma, J. (2024, March). Locality preserving refinement for shape matching with functional maps. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 6, pp. 6207-6215).
25. Xia, Y., & Ma, J. (2026). Locality Optimization Refinement with Deformation for Shape Matching via Functional Maps.International Journal of Computer Vision,134(2), 76.
26. Yang, Y., & Zhang, S. (2024). Pixelwise calibration method for a telecentric structured light system.Applied Optics,63(10), 2562-2569.
27. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., et al. (2023). Segment anything. In Proceedings of ICCV.
28. Li, Q., Liu, Z., Luo, W., Luo, T., & Hou, C. (2026). Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory. arXiv preprint arXiv:2605.24602.
29. Peng, Qucheng, et al. "3d vision-language gaussian splatting." International Conference on Learning Representations. Vol. 2025. 2025.
30. Lin, Z., Guo, W., Gill, N. S., Ritchie, G., Kelly, B., & Song, X. P. (2025). Open cotton boll detection using LiDAR point clouds and RGB images from unmanned aerial systems. Current Plant Biology, 43, 100519.
31. Qi, Z., Yuan, Y., Ruan, X., Wang, S., Zhang, W., & Huang, Q. (2024). Collaborative debias strategy for temporal sentence grounding in video. IEEE Transactions on Circuits and Systems for Video Technology, 34(11), 10972-10986.
32. Wang, Y., Fang, Y., Wang, T., Li, L., Liu, K., Xu, J., ... & Xu, R. (2026). What Limits Vision-and-Language Navigation?.arXiv preprint arXiv:2605.13328.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jacob A. Jenkins
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles in the Journal of Innovative Research and Technology are licensed under Creative Commons Attribution 4.0 (CC BY 4.0). Users can share, adapt, and build upon the material with appropriate credit, without additional restrictions.
