This is mainly for exploring on tiny transformers arch, experiement how to get a small while still powerful transformer architecture.
It was built upon detectron2. We plan to add some of these SOTA transformers based model:
Some more models will be added.
To install, please install detectron2 first, then:
python setup.py build develop
to be done.
All rights reserved by Lucas Jin @ 2022.