Specific surface area is an important parameter to evaluate the capture capacity of covalent-organic frameworks (COFs). Its prediction is critical to theoretical design of new COFs; however, existing computational codes can only provide a rough estimation. Herein, we propose to predict the Brunauer-Emmett-Teller (BET) specific surface areas for COFs using a newly developed deep learning model (COFNet). This model integrates deep learning algorithms with attention mechanism, and innovatively accepts structural images of COFs and the statistical features computed from these images as model inputs. In this study, both model feature extraction and statistical feature computation are simply completed using images only, avoiding additional complex theoretical calculations. This greatly facilitates the prediction of BET specific surface areas. Results show that the proposed COFNet can satisfactorily predict specific surface area of COFs with a Pearson correlation coefficient (R) of 0.812. It significantly outperforms the publicly available Zeo++ software (which achieves R of 0.377). The developed COFNet model is a promising tool to efficiently predict experimental BET specific surface areas of COFs.