536 Commits

Author	SHA1	Message	Date
Sean Sube	988088d64e	quit workers on keyboard signal	2023-02-27 22:52:43 -06:00
Sean Sube	953e5abd36	handle empty errors	2023-02-27 22:45:29 -06:00
Sean Sube	136759285d	set queue timeouts	2023-02-27 22:37:43 -06:00
Sean Sube	0793b61c3a	consistently pass job key to workers	2023-02-27 22:25:53 -06:00
Sean Sube	06f06f5a11	error handling in all threads	2023-02-27 19:48:51 -06:00
Sean Sube	113ad05293	typo	2023-02-27 17:36:26 -06:00
Sean Sube	2327b24022	join all threads	2023-02-27 17:35:31 -06:00
Sean Sube	66a20e60fe	run logger in a thread, clean up status	2023-02-27 17:14:53 -06:00
Sean Sube	13395933dc	always put progress in active jobs	2023-02-26 20:41:16 -06:00
Sean Sube	a37d1a4550	use progress queue	2023-02-26 20:37:22 -06:00
Sean Sube	401ee20526	fix finished flag	2023-02-26 20:13:16 -06:00
Sean Sube	525ee24e91	track started and finished jobs	2023-02-26 20:09:42 -06:00
Sean Sube	eb82e73e59	initialize list of finished jobs	2023-02-26 15:26:54 -06:00
Sean Sube	b931da1d2c	fix imports, lint	2023-02-26 15:21:58 -06:00
Sean Sube	85118d17c6	clear worker flags between jobs, attempt to record finished jobs again	2023-02-26 15:06:40 -06:00
Sean Sube	d1961afdbc	re-implement cancellation	2023-02-26 14:36:32 -06:00
Sean Sube	584dddb5d6	lint all the new stuff	2023-02-26 14:15:30 -06:00
Sean Sube	b880b7a121	set process titles, terminate workers	2023-02-26 13:09:24 -06:00
Sean Sube	6502e1e3c8	recycle worker pool after 10 jobs	2023-02-26 12:58:38 -06:00
Sean Sube	e0737e9e08	update progress and finished flag from worker	2023-02-26 12:51:11 -06:00
Sean Sube	f115326da7	apply patches within workers	2023-02-26 12:32:48 -06:00
Sean Sube	e1d0ad54b7	lock per worker, torch before ORT	2023-02-26 12:24:51 -06:00
Sean Sube	d765a6f01b	make logger start up well	2023-02-26 11:16:33 -06:00
Sean Sube	6998e8735c	rejoin worker pool	2023-02-26 10:47:31 -06:00
Sean Sube	06c74a7a96	feat(api): remove Flask app from global scope	2023-02-26 10:15:12 -06:00
Sean Sube	943281feb5	wire up worker jobs	2023-02-25 23:55:30 -06:00
Sean Sube	f898de8c54	background workers, logger	2023-02-25 23:49:39 -06:00
Sean Sube	e46a1e5fd0	begin switching to per-device torch mp workers	2023-02-25 23:16:32 -06:00
Sean Sube	e03b637f54	fix(api): only run CUDA GC on CUDA devices	2023-02-25 22:32:01 -06:00
Sean Sube	cb8e9e7080	fix scheduler caching	2023-02-25 13:14:34 -06:00
Sean Sube	18f59f034d	avoid passing None as encoder	2023-02-25 13:12:58 -06:00
Sean Sube	3626d69f40	fix(api): correctly handle missing inversion param	2023-02-25 12:54:51 -06:00
Sean Sube	973b0a9352	add context arg	2023-02-25 12:13:22 -06:00
Sean Sube	d88f13cbd7	lint LoRA code and extras, replace public paths with context ones	2023-02-25 12:12:10 -06:00
Sean Sube	ce74183e97	blend LoRAs into existing model	2023-02-25 11:53:43 -06:00
Sean Sube	e8b5ff250d	add none option to inversion menu	2023-02-25 11:53:37 -06:00
Sean Sube	7ad8385c5b	load from encoder path, add default param	2023-02-25 11:53:34 -06:00
Sean Sube	2e7de16778	feat: add a way to select textual inversions	2023-02-25 11:53:32 -06:00
Sean Sube	45f5fca383	use correct fn to make dirs	2023-02-25 11:53:29 -06:00
Sean Sube	3dfaef041e	fix devices, make subdir	2023-02-25 11:53:26 -06:00
Sean Sube	3f4b3fa322	load CLIP on training device	2023-02-25 11:53:23 -06:00
Sean Sube	6b4ced2608	load inversions from extras file	2023-02-25 11:53:20 -06:00
Sean Sube	9dedfc7b28	add experimental LoRA blender	2023-02-25 11:53:17 -06:00
Sean Sube	a31f7b9e1f	feat(api): convert Textual Inversion weights	2023-02-25 11:53:13 -06:00
Sean Sube	947a1bfdec	fix variable names	2023-02-25 11:26:10 -06:00
Sean Sube	44393e3770	feat(api): add txt2txt endpoint	2023-02-25 11:20:36 -06:00
Sean Sube	fd013c88ef	fix(api): enable optimizations during model conversion	2023-02-23 18:43:49 -06:00
Sean Sube	14c7e6a78c	apply lint	2023-02-21 08:26:09 -06:00
Sean Sube	82016a163a	restore single output in chain pipeline stages	2023-02-21 08:14:04 -06:00
Sean Sube	33ab23a474	convert to list of outputs	2023-02-20 23:47:43 -06:00