Optical metasurfaces for general vision processing on the edge

References

Shanahan, M., McDonell, K. & Reynolds, L. Role play with large language models. Nature 623 , 493–498 (2023).

Article

ADS

CAS

PubMed

Google Scholar

Singhal, K. et al. Large language models encode clinical knowledge. Nature 620 , 172–180 (2023).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15 , 102–114 (2021).

Article

ADS

CAS

Google Scholar

Bernstein, L. et al. Single-shot optical neural network. Sci. Adv. 9 , eadg7904 (2023).

Article

CAS

PubMed

PubMed Central

Google Scholar

Zheng, H. et al. Multichannel meta-imagers for accelerating machine vision. Nat. Nanotechnol. 19 , 471–478 (2024).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Zheng, H. et al. Meta-optic accelerators for object classifiers. Sci. Adv. 8 , eabo6410 (2022).

Article

PubMed

PubMed Central

Google Scholar

Luo, M. et al. Meta-optics based parallel convolutional processing for neural network accelerator. Laser Photonics Rev. 18 , 2300984 (2024).

Article

ADS

Google Scholar

Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat. Electron. 5 , 113–122 (2022).

Article

Google Scholar

Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photon.   11 , 441–446 (2017).

Article

ADS

CAS

Google Scholar

Ashtiani, F., Geers, A. J. & Aflatouni, F. An on-chip photonic deep neural network for image classification. Nature 606 , 501–506 (2022).

Article

ADS

CAS

PubMed

Google Scholar

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021).

Article

ADS

CAS

PubMed

Google Scholar

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018).

Article

ADS

MathSciNet

CAS

PubMed

Google Scholar

Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 , 367–373 (2021).

Article

ADS

CAS

Google Scholar

Antonik, P., Marsal, N., Brunner, D. & Rontani, D. Human action recognition with a large-scale brain-inspired photonic computer. Nat. Mach. Intell. 1 , 530–537 (2019).

Article

Google Scholar

Wang, T. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photon.   17 , 408–415 (2023).

Article

ADS

CAS

Google Scholar

Xia, F. et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nat. Photon.   18 , 1067–1075 (2024).

Article

ADS

CAS

Google Scholar

Luo, X. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci. Appl. 11 , 158 (2022).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Huang, C. et al. A silicon photonic–electronic neural network for fibre nonlinearity compensation. Nat. Electron. 4 , 837–844 (2021).

Article

CAS

Google Scholar

Fu, T. et al. Photonic machine learning with on-chip diffractive optics. Nat. Commun. 14 , 70 (2023).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Dong, B. et al. Partial coherence enhances parallelized photonic computing. Nature 632 , 55–62 (2024).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Xu, Z. et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384 , 202–209 (2024).

Article

ADS

CAS

PubMed

Google Scholar

McMahon, P. L. The physics of optical computing. Nat. Rev. Phys. 5 , 717–734 (2023).

Article

Google Scholar

Yildirim, M., Dinc, N. U., Oguz, I., Psaltis, D. & Moser, C. Nonlinear processing with linear optics. Nat. Photon.   18 , 1076–1082 (2024).

Article

ADS

CAS

Google Scholar

Goi, E. et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip. Light Sci. Appl. 10 , 40 (2021).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Chen, Y. et al. All-analog photoelectronic chip for high-speed vision tasks. Nature 623 , 48–57 (2023).

Article

ADS

CAS

PubMed

PubMed Central

Google Scholar

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47 (2020).

Article

ADS

CAS

PubMed

Google Scholar

Feng, H. et al. Integrated lithium niobate microwave photonic processing engine. Nature 627 , 80–87 (2024).

Article

ADS

CAS

PubMed

Google Scholar

Xu, X. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021).

Article

ADS

CAS

PubMed

Google Scholar

Liu, Z. et al. Swin Transformer: hierarchical vision transformer using shifted windows. In Proc. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) 10012–10022 (IEEE, 2021).

Cui, K. et al. Spec…