Commit Graph

21 Commits

Author SHA1 Message Date
Jong Wook Kim 504cf9e491 test fix 2021-07-18 20:28:30 -07:00
Romain Beaumont a2737ac264
Add truncate option to tokenize (#126)
* Add truncate_text option to tokenize

This makes it possible to run tokenize on texts that are longer than the number of tokens
that fit the context length without having to try to guess how to cut in number of 
characters beforehand

* add doc, rename to just "truncate", use eot_token

Co-authored-by: Jong Wook Kim <jongwook@openai.com>
2021-07-18 20:17:40 -07:00
Jong Wook Kim db20393f4a Using non-JIT by default; compat fix with 1.8+ 2021-07-18 18:45:21 -07:00
Jong Wook Kim cfcffb90e6 add YFCC100M subset information
(#50)
2021-04-11 02:29:52 -07:00
Jong Wook Kim 8a665a683d fixed model loading issue (#66) 2021-03-23 03:05:17 -04:00
Jong Wook Kim 290ac5cb15
Correctly initializing the logit scale parameter
adding numpy import
2021-03-22 22:09:57 -04:00
Jong Wook Kim 43c953e231
Correctly initializing the logit scale parameter
cf. #46
2021-03-22 18:07:08 -04:00
Smittyvb beba48f353
Update models URIs in notebook (#52) 2021-03-07 21:58:54 -05:00
Jong Wook Kim fd6c1443c2 add RN101 and RN50x4; update paper URL and model card 2021-03-04 12:30:39 -05:00
Sebastian Berns 4c0275784d
Load model from path (#41)
* Load model from path

* showing download progress in "MiB"

* clip.load() now can take a file path

Co-authored-by: Jong Wook Kim <jongwook@openai.com>
2021-02-16 06:19:42 -05:00
Jong Wook Kim 8f6deb52a1 showing download progress in "MiB" 2021-02-15 22:00:33 -05:00
Jong Wook Kim e5347713f4 add prompt engineering notebook 2021-01-30 16:54:01 +09:00
Jong Wook Kim d1cf43d17e using compatible version specifier in requirements.txt 2021-01-30 06:33:56 +09:00
Jong Wook Kim c42a8e3c9e added parameter initialization (fixes #15) 2021-01-30 03:26:10 +09:00
Jong Wook Kim 4a443c18a4 updated README.md to include pip install command 2021-01-30 03:09:59 +09:00
boba_and_beer 3bee28119e
Make the repo installable as a package (#26) 2021-01-30 03:05:01 +09:00
Jong Wook Kim 578a1d3e2e moved the notebook to subfolder 2021-01-30 00:22:03 +09:00
Jong Wook Kim 6bc0bd8873 added RN50 checkpoint and non-JIT model implementation 2021-01-13 08:35:50 +09:00
Jong Wook Kim ebd0e35aac correctly tokenizing SOT/EOT tokens (fixes #8) 2021-01-09 02:55:09 +09:00
Jong Wook Kim c89e0c16de
Update the paper URL 2021-01-07 04:38:27 +09:00
Jong Wook Kim b1c4b6be58 initial commit 2021-01-06 04:06:12 +09:00