<div class="csl-bib-body">
<div class="csl-entry">Wang, R., Wang, X., Xu, Z., Xu, W., Chen, J., & Lukasiewicz, T. (2023). MvCo-DoT: Multi-View Contrastive Domain Transfer Network for Medical Report Generation. In <i>ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)</i>. 2023 International Conference on Acoustics, Speech, and Signal Processing, Rhodes, Greece. IEEE. https://doi.org/10.1109/ICASSP49357.2023.10095254</div>
</div>
-
dc.identifier.uri
http://hdl.handle.net/20.500.12708/192514
-
dc.description.abstract
In clinical scenarios, multiple medical images with different views are usually generated at the same time, and they have high semantic consistency. However, the existing medical report generation methods cannot exploit the rich multi-view mutual information of medical images. Therefore, in this work, we propose the first multi-view medical report generation model, called MvCo-DoT. Specifically, MvCo-DoT first propose a multi-view contrastive learning (MvCo) strategy to help the deep reinforcement learning based model utilize the consistency of multi-view inputs for better model learning. Then, to close the performance gaps of using multi-view and single-view inputs, a domain transfer network is further proposed to ensure MvCo-DoT achieve almost the same performance as multi-view inputs using only single-view inputs. Extensive experiments on the IU X-Ray public dataset show that MvCo-DoT outperforms the SOTA medical report generation baselines in all metrics.
en
dc.language.iso
en
-
dc.subject
medical report generation
en
dc.subject
multi-view medical report generation
en
dc.title
MvCo-DoT: Multi-View Contrastive Domain Transfer Network for Medical Report Generation
en
dc.type
Inproceedings
en
dc.type
Konferenzbeitrag
de
dc.relation.publication
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
-
dc.contributor.affiliation
Hebei University of Technology, China
-
dc.contributor.affiliation
Hebei University of Technology, China
-
dc.contributor.affiliation
Hebei University of Technology, China
-
dc.contributor.affiliation
Hebei University of Technology, China
-
dc.contributor.affiliation
Shenzhen University, China
-
dc.relation.isbn
978-1-7281-6327-7
-
dc.relation.doi
10.1109/ICASSP49357.2023
-
dc.relation.issn
1520-6149
-
dc.type.category
Full-Paper Contribution
-
dc.relation.eissn
2379-190X
-
tuw.booktitle
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)