🔬 SpecSolver

Solving Spatial–Spectral Fusion via Semantic Transformer

Wei Li¹, Junwei Zhu¹, Honghui Xu¹, Jiawei Jiang¹, Jianwei Zheng^1✉️
1Zhejiang University of Technology
✉️ Corresponding author

---

🚀 ACMMM 2025 News (2025-07-05)

🎉 Exciting Announcement! SpecSolver has been officially accepted to ACM Multimedia (ACMMM) 2025 (conference paper). Our open-source repository is under active development—stay tuned for the camera-ready paper, code releases, and pretrained models!

---

📋 Roadmap & To-dos

• ✅ Publish camera-ready version of the paper and supplementary materials
• ✅ Publication citation format
• ✅ Open-source the complete Train & Test code and pretrained weights
• ✅ Release dataset for reproducible experiments

Tip: ⭐ Star our repository to receive updates on releases and new features.

---

🔍 Introduction

Semantic transformer-based solvers like SpecSolver draw inspiration from superpixel segmentation but overcome its limitations in spatial–spectral fusion (SSF). Our framework:

1. Semantic Slicing: Learns flexible pixel groupings (slices) through a novel Semantic-Attention mechanism, ensuring differentiability and end-to-end training.
2. Token Encoding: Transforms each slice into a Semantic-Superpixel token, capturing rich spatial and spectral cues.
3. Transformer Solver: Applies attention across tokens to model long-range dependencies efficiently, supporting multiple upscaling factors with linear complexity.

Why SpecSolver?

• ⚡ Efficiency: Linear computational cost in the number of pixels
• 🌟 Flexibility: Adaptive slice shapes tuned to semantic content
• 🎯 Accuracy: State-of-the-art performance on standard SSF benchmarks

✨Quick Start

Follow these steps to train and test the SpecSolver models with a scaling factor of 4:

1. Train on CAVE dataset

   python -m Train.SpectralSolver_Train_cave --sf 4
   

2. Test on CAVE dataset

   python -m Test.SpectralSolver_Test_cave --sf 4
   

3. Train on Harvard dataset

   python -m Train.SpectralSolver_Train_Harvard --sf 4
   

4. Test on Harvard dataset

   python -m Test.SpectralSolver_Test_harvard --sf 4
   

---

📊 Public Datasets

Dataset	Download Link	Extraction Code
CAVE	⬇️ Download CAVE Dataset	dju8
Harvard	⬇️ Download Harvard Dataset	aque

💡 Tip:
Your folder structure should look like:

./Cavedataset/
├── Train
└── Test

---

📚 Citation

If SpecSolver contributes to your research, please cite:

@inproceedings{li2025specsolver,
  title={SpecSolver: Solving Spatial-Spectral Fusion via Semantic Transformer},
  author={Li, Wei and Zhu, Junwei and Xu, Honghui and Jiang, Jiawei and Zheng, Jianwei},
  booktitle={Proceedings of the 33rd ACM International Conference on Multimedia},
  pages={1607--1616},
  year={2025}
}