PMID- 36445993
OWN - NLM
STAT- PubMed-not-MEDLINE
DCOM- 20230606
LR  - 20230606
IS  - 1939-3539 (Electronic)
IS  - 0098-5589 (Linking)
VI  - 45
IP  - 7
DP  - 2023 Jul
TI  - Spatial-Temporal Transformer for Video Snapshot Compressive Imaging.
PG  - 9072-9089
LID - 10.1109/TPAMI.2022.3225382 [doi]
AB  - Video snapshot compressive imaging (SCI) captures multiple sequential video 
      frames by a single measurement using the idea of computational imaging. The 
      underlying principle is to modulate high-speed frames through different masks and 
      these modulated frames are summed to a single measurement captured by a low-speed 
      2D sensor (dubbed optical encoder); following this, algorithms are employed to 
      reconstruct the desired high-speed frames (dubbed software decoder) if needed. In 
      this article, we consider the reconstruction algorithm in video SCI, i.e., 
      recovering a series of video frames from a compressed measurement. Specifically, 
      we propose a Spatial-Temporal transFormer (STFormer) to exploit the correlation 
      in both spatial and temporal domains. STFormer network is composed of a token 
      generation block, a video reconstruction block, and these two blocks are 
      connected by a series of STFormer blocks. Each STFormer block consists of a 
      spatial self-attention branch, a temporal self-attention branch and the outputs 
      of these two branches are integrated by a fusion network. Extensive results on 
      both simulated and real data demonstrate the state-of-the-art performance of 
      STFormer. The code and models are publicly available at 
      https://github.com/ucaswangls/STFormer.
FAU - Wang, Lishun
AU  - Wang L
FAU - Cao, Miao
AU  - Cao M
FAU - Zhong, Yong
AU  - Zhong Y
FAU - Yuan, Xin
AU  - Yuan X
LA  - eng
PT  - Journal Article
DEP - 20230605
PL  - United States
TA  - IEEE Trans Pattern Anal Mach Intell
JT  - IEEE transactions on pattern analysis and machine intelligence
JID - 9885960
SB  - IM
EDAT- 2022/11/30 06:00
MHDA- 2022/11/30 06:01
CRDT- 2022/11/29 15:03
PHST- 2022/11/30 06:01 [medline]
PHST- 2022/11/30 06:00 [pubmed]
PHST- 2022/11/29 15:03 [entrez]
AID - 10.1109/TPAMI.2022.3225382 [doi]
PST - ppublish
SO  - IEEE Trans Pattern Anal Mach Intell. 2023 Jul;45(7):9072-9089. doi: 
      10.1109/TPAMI.2022.3225382. Epub 2023 Jun 5.