Mert
21d7cf6de6
chore(server): make date-time logs verbose ( #13659 )
...
make date-time logs verbose
2024-10-21 16:35:56 -05:00
Mert
62e55f3db9
fix(server): Revert "fix(server): copy video projection metadata for 360 videos ( #12376 )" ( #13611 )
...
Revert "fix(server): copy video projection metadata for 360 videos (#12376 )"
This reverts commit aa0097bde2
2024-10-20 10:15:50 -05:00
Mert
23646f0d55
feat(server): vaapi hardware decoding ( #13561 )
...
* add hw decoding for vaapi
* add tests
* update docs
2024-10-17 18:02:41 -04:00
Mert
c266465317
chore(deps): update base-image to v20241016 (major) ( #13512 )
...
chore(deps): update base-image to v20241016
2024-10-16 11:58:51 -04:00
Mert
e7397f35c9
chore(ml): update pydantic ( #13230 )
...
* update pydantic
* fix typing
* remove unused import
* remove unused schema
2024-10-13 18:00:21 -04:00
Mert
3ba2602664
fix(ml): pin onnxruntime-openvino ( #13290 )
2024-10-08 18:03:28 -04:00
Mert
08d428cbce
fix(server): duplicate faces, face insert query failing ( #13294 )
...
fix duplicate faces, query failing
2024-10-08 21:37:41 +00:00
Mert
bd826b0b9b
feat(ml): round-robin device assignment ( #13237 )
...
* round-robin device assignment
* docs and tests
clarify doc
2024-10-07 17:37:45 -04:00
Mert
063969ca05
fix(server): searching with both personIds and withPeople ( #13254 )
...
* use cte
* linting
2024-10-07 15:44:04 -04:00
Mert
4bb42c4a25
docs: update version range for pgvecto.rs ( #13231 )
...
* update pgvecto.rs range
* clarify pg17
2024-10-06 13:16:31 -04:00
Mert
2c87683fd4
feat(server): refresh face detection ( #12335 )
...
* refresh faces
handle non-ml faces
* fix metadata face handling
* updated tests
* added todo comment
2024-10-03 21:58:28 -04:00
Mert
8ee825964d
fix(server): thumbnail colorspace handling ( #13130 )
...
fix colorspace handling
2024-10-02 20:53:17 +00:00
Mert
7adb35e59e
fix(server): /search/random failing with certain options ( #13040 )
...
* fix relation handling, remove pagination
* update api, sql
* update mock
2024-09-30 00:29:35 -04:00
Mert
2f13db51df
fix(server): "all" button for facial recognition deleting faces instead of unassigning them ( #13042 )
...
* unassign faces instead of deleting them
* formatting
2024-09-30 00:29:14 -04:00
Mert
9b309e84c9
docs: update config file ( #13041 )
...
update config file
2024-09-30 11:11:42 +07:00
Mert
2bcd27e166
feat(server): generate all thumbnails for an asset in one job ( #13012 )
...
* wip
cleanup
add success logs, rename method
do thumbhash too
fixes
fix tests
handle `notify`
wip refactor
refactor
* update tests
* update sql
* pr feedback
* remove unused code
* formatting
2024-09-28 17:47:24 +00:00
Mert
995f0fda47
feat(server): separate quality for thumbnail and preview images ( #13006 )
...
* allow different thumbnail and preview quality, better config structure
* update web and api
* wording
* remove empty line?
2024-09-28 06:01:04 +00:00
Mert
4248594ac5
feat(server): better transcoding logs ( #13000 )
...
* better transcoding logs
* pr feedback
2024-09-27 18:10:39 -04:00
Mert
a68e6be7e1
chore(ml): remove deprecated kwarg when downloading models ( #12589 )
...
remove local_dir_use_symlinks
2024-09-11 22:21:33 -05:00
Mert
22dc9bcebb
fix(ml): batch axis not being added for recognition model ( #12588 )
...
* fix has_batch_axis
* fix typing
2024-09-11 22:21:12 -05:00
Mert
ad58d7e23e
chore(ml): downgrade to cuda 12.2 ( #12587 )
...
* downgrade to cuda 12.2
* update docs
2024-09-12 00:25:57 +00:00
Mert
56bf3cc3d1
chore(ml): bump intel driver version ( #12455 )
...
update to 24.31.30508.7
2024-09-07 23:08:11 -04:00
Mert
aa0097bde2
fix(server): copy video projection metadata for 360 videos ( #12376 )
2024-09-06 00:30:34 -04:00
Mert
eb7777639d
fix(server): clean face tables after delete ( #12375 )
...
clean face tables after delete
2024-09-05 23:09:19 +00:00
Mert
036676d501
fix(ml): tokenization for webli models ( #11881 )
2024-08-18 11:05:10 -04:00
Mert
23d4314eed
chore(server): support pgvecto.rs 0.3.0 ( #11624 )
...
relax pgvecto.rs constraint
2024-08-06 23:04:55 -04:00
Mert
d5b23373c7
refactor(server): startup checks for vector extension ( #11559 )
...
* update update logic
refactor
* update tests
* get version range through repo method, make tests more static
* move "should work" test
2024-08-05 21:00:25 -04:00
Mert
4ed75f2ac9
refactor(server): add config events for clip ( #11575 )
...
use config events for clip, add tests
formatting
2024-08-04 21:00:36 +00:00
Mert
3f4b783889
chore: add healthcheck field to server and ml ( #11573 )
...
add healthcheck field to server and ml
2024-08-04 13:37:43 -05:00
Mert
41580696c7
feat(ml): add more search models ( #11468 )
...
* update export code
* add uuid glob, sort model names
* add new models to ml, sort names
* add new models to server, sort by dims and name
* typo in name
* update export dependencies
* onnx save function
* format
2024-07-31 04:34:45 +00:00
Mert
9d2d556200
feat(server): accepted video containers ( #11274 )
...
* add accepted container config
* update api
* mp4 option makes no sense
* add to transcoding settings
* wording
* updated spec config
* formatting
2024-07-21 21:14:23 +00:00
Mert
173c9070c8
fix(ml): re-add worker env ( #11192 )
...
re-add worker env
2024-07-18 17:50:52 +00:00
Mert
88f62087fd
chore(ml): set higher worker timeout for openvino ( #11174 )
2024-07-18 10:50:57 -04:00
Mert and Alex
cc1235d4aa
docs: facial recognition and general clean-up ( #11106 )
...
* add facial recognition docs, clean up existing info
* Update smart-search.md
Co-authored-by: Alex <alex.tran1502@gmail.com >
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-07-14 21:08:16 -05:00
Mert
8193416230
feat(server): conditionally run facial recognition nightly ( #11080 )
...
* only run nightly if new person
* add tests
* use string instead of date
* update sql
* update tests
* simplify condition
2024-07-14 22:53:42 +00:00
Mert
f43721ec92
fix(ml): armnn not being used ( #10929 )
...
* fix armnn not being used, move fallback handling to main, add tests
* formatting
2024-07-10 09:20:43 -05:00
Mert
0b88bef157
fix(server): face search migration sometimes failing ( #10827 )
...
* turn it off and back on
* handle missing smart search embedding column
* handle missing face embedding column
* simplify
* Revert "simplify"
This reverts commit 8322af0baf
2024-07-03 21:02:43 -05:00
Mert
6356c28f64
refactor(ml): model sessions ( #10559 )
2024-06-25 12:00:24 -04:00
Mert
42f3b50422
fix(server): /places entries sometimes not ordered alphabetically ( #10514 )
2024-06-20 23:48:19 -04:00
Mert
a42af06889
fix(ml): limit load retries ( #10494 )
2024-06-20 14:13:18 -04:00
Mert
79a8ab71ef
fix(server): reindex after changing to a model with a different dimension size ( #10496 )
...
reindex after truncating
2024-06-19 17:25:02 -04:00
Mert
1191978d50
fix(server): library refresh not checking trashed assets ( #10495 )
...
* set `withDeleted`
* update sql
2024-06-19 20:42:55 +00:00
Mert
b3f9641edf
feat(web): bulk deduplicate ( #10448 )
...
* bulk deduplicate
* notification for keeping all duplicates
* fix notification
* remove unused text
* pr feedback
* wording
* formatting
2024-06-19 12:11:59 -04:00
Mert
86cbc6e125
chore(ml): support python 3.12 ( #10481 )
2024-06-19 10:51:10 -04:00
Mert
968553a50e
fix(server): video thumbnail generation failing with single i-frame ( #10477 )
2024-06-19 10:50:25 -04:00
Mert
5813dc02d1
fix(server): let thumbnail generation fail on error ( #10479 )
2024-06-19 10:50:09 -04:00
Mert
58b17a866b
feat(web): display original heif images for safari ( #10478 )
2024-06-19 10:49:59 -04:00
Mert
e58131492d
fix(server): consider all I-frames for video thumbnails ( #10471 )
...
nointra instead of nokey
2024-06-18 19:02:33 -04:00
Mert
6b1b5054f8
feat(server): separate face search relation ( #10371 )
...
* wip
* various fixes
* new migration
* fix test
* add face search entity, update sql
* update e2e
* set storage to external
2024-06-16 19:25:27 +00:00
Mert
e77e87b936
fix(server): orientation handling for person thumbnails ( #10382 )
...
fix orientation handling
2024-06-16 08:45:58 -07:00
Mert
fb641c74be
fix(server): use preview image when generating person thumbnail from video ( #10240 )
2024-06-12 22:16:26 -04:00
Mert
5dda5d93f5
chore(docs): remove microservices from hwa docs ( #10188 )
...
remove microservices from hwa docs
2024-06-12 11:57:40 +01:00
Mert
cdc98de848
fix(server): increase pixel limit for thumbnail generation ( #10181 )
...
disable input limit
2024-06-11 22:11:03 -04:00
Mert
126cbeabe8
feat(server): add av1 support for vaapi ( #10180 )
...
add av1
2024-06-12 00:24:06 +00:00
Mert
2e0c6f6fff
fix: postgres health check reporting any db without checksums as unhealthy ( #10178 )
...
handle disabled checksumming
2024-06-12 00:18:24 +00:00
Mert
df412d60c5
fix(server): min face detection score not being passed correctly ( #10050 )
...
fix min score not being passed correctly
2024-06-08 13:55:19 -04:00
Mert
62f8bd80f4
fix(server): add fallback for video thumbnail generation ( #10034 )
...
they called me a madman
2024-06-08 04:55:05 -05:00
Mert
f2148ddf03
fix(server): video thumbnail colors when using webp ( #10018 )
...
use gbrpf32le
2024-06-07 02:43:10 -04:00
Mert
2b1b43a7e4
feat(ml): composable ml ( #9973 )
...
* modularize model classes
* various fixes
* expose port
* change response
* round coordinates
* simplify preload
* update server
* simplify interface
simplify
* update tests
* composable endpoint
* cleanup
fixes
remove unnecessary interface
support text input, cleanup
* ew camelcase
* update server
server fixes
fix typing
* ml fixes
update locustfile
fixes
* cleaner response
* better repo response
* update tests
formatting and typing
rename
* undo compose change
* linting
fix type
actually fix typing
* stricter typing
fix detection-only response
no need for defaultdict
* update spec file
update api
linting
* update e2e
* unnecessary dimension
* remove commented code
* remove duplicate code
* remove unused imports
* add batch dim
2024-06-07 03:09:47 +00:00
Mert
8812c3afcf
fix(server): apply qsv and vaapi quality to video stream only ( #9807 )
...
apply quality to video stream only
2024-05-28 09:49:51 +01:00
Mert
fbc3790cb6
fix(server): video thumbnail generation failing when using qsv ( #9808 )
...
fix multiplier
2024-05-28 09:47:41 +01:00
Mert
4c7347d653
fix: re-add extends section for server in Compose files ( #9806 )
...
re-add extends section
2024-05-27 21:04:07 +00:00
Mert
dca420ef70
chore: refactor transcode config routing ( #9800 )
...
* chore: refactor transcode config
* rename parameter
* handle no /dev/dri
* prefer undefined
2024-05-27 15:20:07 -04:00
Mert
21bd20fd75
fix(server): nvenc not working when there are no filters ( #9802 )
...
don't add format=nv12
2024-05-27 15:18:01 -04:00
Mert
351dd647a9
feat(server): better video thumbnails ( #9784 )
2024-05-27 12:08:38 -04:00
Mert and Alex
e7c8501930
fix(server): search duplicates of the same asset type ( #9747 )
...
* search by type
* make sql
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-05-26 22:04:23 +00:00
Mert
69b5eb005f
fix(server): use qsv format for hwmap ( #9722 )
...
use qsv format for hwmap
2024-05-24 09:50:28 +01:00
Mert
a5e8b451b2
feat(server): qsv hardware decoding and tone-mapping ( #9689 )
...
* qsv hw decoding and tone-mapping
* fix vaapi
* add tests
* formatting
* handle device name without path
2024-05-23 03:58:29 +00:00
Mert
9aed736911
fix(ml): openvino not working with kernel 6.7.5 or later ( #9541 )
...
* add envs
* move to Dockerfile
2024-05-18 00:00:20 +00:00
Mert
ff52300624
refactor(server): duplicate controller and service ( #9542 )
...
* duplicate controller and service
* change endpoint name
* fix search tests
* remove unused import
* add to index
2024-05-16 19:39:33 -04:00
Mert
d8eca168ca
feat(server): fully accelerated nvenc ( #9452 )
...
* use arrayContaining
* libplacebo for nvenc
update dockerfile
* tweaks
* update nvenc options
* tweak settings
* refactor
* toggle for hardware decoding, software / hardware decoding for nvenc and rkmpp
* fix software tone-mapping not being applied
* separate configs for hw/sw
* update api
* add hw decode toggle
* fix mutating config
* remove `version` flag
* fix config type
* remove submodule
* handle temporal AQ
* remove duplicate tests
* use `tonemap_opencl`
* wording
* update docs
2024-05-16 13:30:26 -04:00
64636c0618
feat(server): near-duplicate detection ( #8228 )
...
* duplicate detection job, entity, config
* queueing
* job panel, update api
* use embedding in db instead of fetching
* disable concurrency
* only queue visible assets
* handle multiple duplicateIds
* update concurrent queue check
* add provider
* add web placeholder, server endpoint, migration, various fixes
* update sql
* select embedding by default
* rename variable
* simplify
* remove separate entity, handle re-running with different threshold, set default back to 0.02
* fix tests
* add tests
* add index to entity
* formatting
* update asset mock
* fix `upsertJobStatus` signature
* update sql
* formatting
* default to 0.03
* optimize clustering
* use asset's `duplicateId` if present
* update sql
* update tests
* expose admin setting
* refactor
* formatting
* skip if ml is disabled
* debug trash e2e
* remove from web
* remove from sidebar
* test if ml is disabled
* update sql
* separate duplicate detection from clip in config, disable by default for now
* fix doc
* lower minimum `maxDistance`
* update api
* Add and Use Duplicate Detection Feature Flag (#9364 )
* Add Duplicate Detection Flag
* Use Duplicate Detection Flag
* Attempt Fixes for Failing Checks
* lower minimum `maxDistance`
* fix tests
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com >
* chore: fixes and additions after rebase
* chore: update api (remove new Role enum)
* fix: left join smart search so getAll works without machine learning
* test: trash e2e go back to checking length of assets is zero
* chore: regen api after rebase
* test: fix tests after rebase
* redundant join
---------
Co-authored-by: Nicholas Flamy <30300649+NicholasFlamy@users.noreply.github.com >
Co-authored-by: Zack Pollard <zackpollard@ymail.com >
Co-authored-by: Zack Pollard <zack@futo.org >
2024-05-16 18:08:37 +01:00
Mert
ad915ccd64
docs(ml): update link ( #9477 )
...
update link
2024-05-14 15:34:36 +01:00
Mert
f667c9597b
chore(server): less rigid transcoding tests ( #9401 )
...
* use arrayContaining
* clean up
2024-05-12 00:04:31 -04:00
Mert
bb4843747b
perf: cache transcoding devices ( #9381 )
...
cache transcoding devices
2024-05-10 15:03:47 -04:00
Mert and Daniel Dietzler
f3fbb9b588
perf: cache getConfig ( #9377 )
...
* cache `getConfig`
* critical fix
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
---------
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
2024-05-10 18:15:25 +00:00
Mert and Alex
1167f0f2b7
feat(server): optimize person thumbnail generation ( #7513 )
...
* do crop and resize together
* redundant `pipelineColorspace` call
* formatting
* fix rebase
* handle orientation
* remove unused import
* formatting
* use oriented dimensions for half size calculation
* default case for orientation
* simplify orientation code
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-05-08 08:09:34 -05:00
Mert
a0c43a2b5a
fix(server): person thumbnails not being queued during thumbnail generation ( #9195 )
...
use face id instead of asset id
2024-05-02 01:33:17 +01:00
Mert
2648032163
fix(server): untracked thumbnail and preview images ( #9126 )
...
* delete old thumbnails
* add tests
* revert invisible asset handling
2024-04-27 18:43:05 -04:00
Mert
d52ed51aab
fix(cli): dry run being inaccurate ( #9088 )
2024-04-25 21:14:48 -04:00
Mert
c14a2eda5d
fix(cli): --ignore getting ignored ( #9071 )
2024-04-25 10:48:17 -04:00
Mert
a90138e42e
docs(ml): clarify cuda and openvino requirements ( #9072 )
...
update docs
2024-04-24 23:15:26 -05:00
Mert
466451abc9
feat(server): env variable to skip migrations on startup ( #9069 )
...
* env variable to skip migrations
* update docs
* update env doc
2024-04-24 22:52:38 -04:00
Mert
c30cd3b378
chore: test more formats in e2e ( #9001 )
2024-04-22 01:35:27 -04:00
Mert
fd4514711f
feat(server): enable AV1 encoding for NVENC ( #8959 )
...
allow av1 for nvenc
2024-04-20 14:52:50 -04:00
Mert and Alex Tran
431ffebddd
feat(server): use embedded preview from raw images ( #8773 )
...
* extract embedded
* update api
* add tests
* move temp file logic outside of media repo
* formatting
* revert `toSorted`
* disable by default
* clarify setting description
* wording
* wording
* update docs
* check extracted image dimensions
* test that it unlinks
* formatting
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
2024-04-19 10:50:13 -05:00
Mert
596c35dc00
fix(server): skip invisible assets for thumbnail generation and ml ( #8891 )
...
* skip invisible assets for thumbnail generation and ml
* no need to update job status
* fix thumbhash check order
* linting
2024-04-19 01:37:55 +00:00
Mert
8573c84605
fix(server): include archived images in face detection ( #8892 )
2024-04-17 23:47:24 -04:00
Mert
8c9a092561
docs(ml): update hardware acceleration doc ( #8700 )
...
* update docs
* formatting
2024-04-11 09:39:18 +02:00
Mert
e13d4c9c13
chore: add code owner ( #8701 )
2024-04-11 06:34:19 +00:00
Mert
105a74caca
feat(server,web): configure image format ( #8581 )
2024-04-07 12:44:34 -04:00
Mert
55b9acca78
fix(server): hevc tag being set when copying a non-hevc stream ( #8582 )
2024-04-07 12:44:09 -04:00
Mert
0d130b8957
fix(server): x264/x265 params not being set correctly ( #8587 )
2024-04-07 12:43:50 -04:00
Mert
0aa5d3daeb
fix(web): reset to default button always being shown ( #8577 )
2024-04-07 12:43:40 -04:00
Mert
335c03d0b8
chore(server): better typing for system config key ( #8580 )
...
* config type safety
* typeorm fix
* typing fixes
* don't use enum in db
* add todo
2024-04-07 01:47:33 +00:00
Mert
4681ff88d0
fix(server): image config not being updated ( #8579 )
...
update system config key
2024-04-06 21:06:26 -04:00
Mert
33fd27f113
fix(web): some settings not disabled when using config file ( #8576 )
...
add disable
2024-04-06 20:59:00 -04:00
Mert and Alex Tran
8edc2fb46f
refactor(server): decouple generated images from image formats ( #8246 )
...
* rename
thumbnail config
update target paths, fix tests
rename to image settings
replace legacy enum
better typing
update sql
update api
remove config option
fix
* update docs
* update other thumbnail configs in migration
* keep legacy enum for now
* fix jumbled job names
* fix jumbled job names in tests
* rename thumbhash job
* rename dto
* fix tests
* preserve order
* remove unused import
* keep old fields in dto, marked deprecated
* update sql
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
2024-04-02 04:56:56 +00:00
Mert
169d9d18b0
docs: document metric env variables, add job metric env ( #8406 )
...
* update env docs
* show options
2024-03-31 17:29:11 +00:00
Mert
5bc9158724
fix(server): penalize null geodata fields when searching places ( #8408 )
2024-03-31 10:59:11 -04:00
Mert
