Mert
036676d501
fix(ml): tokenization for webli models ( #11881 )
2024-08-18 11:05:10 -04:00
Mert
23d4314eed
chore(server): support pgvecto.rs 0.3.0 ( #11624 )
...
relax pgvecto.rs constraint
2024-08-06 23:04:55 -04:00
Mert
d5b23373c7
refactor(server): startup checks for vector extension ( #11559 )
...
* update update logic
refactor
* update tests
* get version range through repo method, make tests more static
* move "should work" test
2024-08-05 21:00:25 -04:00
Mert
4ed75f2ac9
refactor(server): add config events for clip ( #11575 )
...
use config events for clip, add tests
formatting
2024-08-04 21:00:36 +00:00
Mert
3f4b783889
chore: add healthcheck field to server and ml ( #11573 )
...
add healthcheck field to server and ml
2024-08-04 13:37:43 -05:00
Mert
41580696c7
feat(ml): add more search models ( #11468 )
...
* update export code
* add uuid glob, sort model names
* add new models to ml, sort names
* add new models to server, sort by dims and name
* typo in name
* update export dependencies
* onnx save function
* format
2024-07-31 04:34:45 +00:00
Mert
9d2d556200
feat(server): accepted video containers ( #11274 )
...
* add accepted container config
* update api
* mp4 option makes no sense
* add to transcoding settings
* wording
* updated spec config
* formatting
2024-07-21 21:14:23 +00:00
Mert
173c9070c8
fix(ml): re-add worker env ( #11192 )
...
re-add worker env
2024-07-18 17:50:52 +00:00
Mert
88f62087fd
chore(ml): set higher worker timeout for openvino ( #11174 )
2024-07-18 10:50:57 -04:00
Mert and Alex
cc1235d4aa
docs: facial recognition and general clean-up ( #11106 )
...
* add facial recognition docs, clean up existing info
* Update smart-search.md
Co-authored-by: Alex <alex.tran1502@gmail.com >
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-07-14 21:08:16 -05:00
Mert
8193416230
feat(server): conditionally run facial recognition nightly ( #11080 )
...
* only run nightly if new person
* add tests
* use string instead of date
* update sql
* update tests
* simplify condition
2024-07-14 22:53:42 +00:00
Mert
f43721ec92
fix(ml): armnn not being used ( #10929 )
...
* fix armnn not being used, move fallback handling to main, add tests
* formatting
2024-07-10 09:20:43 -05:00
Mert
0b88bef157
fix(server): face search migration sometimes failing ( #10827 )
...
* turn it off and back on
* handle missing smart search embedding column
* handle missing face embedding column
* simplify
* Revert "simplify"
This reverts commit 8322af0baf
2024-07-03 21:02:43 -05:00
Mert
6356c28f64
refactor(ml): model sessions ( #10559 )
2024-06-25 12:00:24 -04:00
Mert
42f3b50422
fix(server): /places entries sometimes not ordered alphabetically ( #10514 )
2024-06-20 23:48:19 -04:00
Mert
a42af06889
fix(ml): limit load retries ( #10494 )
2024-06-20 14:13:18 -04:00
Mert
79a8ab71ef
fix(server): reindex after changing to a model with a different dimension size ( #10496 )
...
reindex after truncating
2024-06-19 17:25:02 -04:00
Mert
1191978d50
fix(server): library refresh not checking trashed assets ( #10495 )
...
* set `withDeleted`
* update sql
2024-06-19 20:42:55 +00:00
Mert
b3f9641edf
feat(web): bulk deduplicate ( #10448 )
...
* bulk deduplicate
* notification for keeping all duplicates
* fix notification
* remove unused text
* pr feedback
* wording
* formatting
2024-06-19 12:11:59 -04:00
Mert
86cbc6e125
chore(ml): support python 3.12 ( #10481 )
2024-06-19 10:51:10 -04:00
Mert
968553a50e
fix(server): video thumbnail generation failing with single i-frame ( #10477 )
2024-06-19 10:50:25 -04:00
Mert
5813dc02d1
fix(server): let thumbnail generation fail on error ( #10479 )
2024-06-19 10:50:09 -04:00
Mert
58b17a866b
feat(web): display original heif images for safari ( #10478 )
2024-06-19 10:49:59 -04:00
Mert
e58131492d
fix(server): consider all I-frames for video thumbnails ( #10471 )
...
nointra instead of nokey
2024-06-18 19:02:33 -04:00
Mert
6b1b5054f8
feat(server): separate face search relation ( #10371 )
...
* wip
* various fixes
* new migration
* fix test
* add face search entity, update sql
* update e2e
* set storage to external
2024-06-16 19:25:27 +00:00
Mert
e77e87b936
fix(server): orientation handling for person thumbnails ( #10382 )
...
fix orientation handling
2024-06-16 08:45:58 -07:00
Mert
fb641c74be
fix(server): use preview image when generating person thumbnail from video ( #10240 )
2024-06-12 22:16:26 -04:00
Mert
5dda5d93f5
chore(docs): remove microservices from hwa docs ( #10188 )
...
remove microservices from hwa docs
2024-06-12 11:57:40 +01:00
Mert
cdc98de848
fix(server): increase pixel limit for thumbnail generation ( #10181 )
...
disable input limit
2024-06-11 22:11:03 -04:00
Mert
126cbeabe8
feat(server): add av1 support for vaapi ( #10180 )
...
add av1
2024-06-12 00:24:06 +00:00
Mert
2e0c6f6fff
fix: postgres health check reporting any db without checksums as unhealthy ( #10178 )
...
handle disabled checksumming
2024-06-12 00:18:24 +00:00
Mert
df412d60c5
fix(server): min face detection score not being passed correctly ( #10050 )
...
fix min score not being passed correctly
2024-06-08 13:55:19 -04:00
Mert
62f8bd80f4
fix(server): add fallback for video thumbnail generation ( #10034 )
...
they called me a madman
2024-06-08 04:55:05 -05:00
Mert
f2148ddf03
fix(server): video thumbnail colors when using webp ( #10018 )
...
use gbrpf32le
2024-06-07 02:43:10 -04:00
Mert
2b1b43a7e4
feat(ml): composable ml ( #9973 )
...
* modularize model classes
* various fixes
* expose port
* change response
* round coordinates
* simplify preload
* update server
* simplify interface
simplify
* update tests
* composable endpoint
* cleanup
fixes
remove unnecessary interface
support text input, cleanup
* ew camelcase
* update server
server fixes
fix typing
* ml fixes
update locustfile
fixes
* cleaner response
* better repo response
* update tests
formatting and typing
rename
* undo compose change
* linting
fix type
actually fix typing
* stricter typing
fix detection-only response
no need for defaultdict
* update spec file
update api
linting
* update e2e
* unnecessary dimension
* remove commented code
* remove duplicate code
* remove unused imports
* add batch dim
2024-06-07 03:09:47 +00:00
Mert
8812c3afcf
fix(server): apply qsv and vaapi quality to video stream only ( #9807 )
...
apply quality to video stream only
2024-05-28 09:49:51 +01:00
Mert
fbc3790cb6
fix(server): video thumbnail generation failing when using qsv ( #9808 )
...
fix multiplier
2024-05-28 09:47:41 +01:00
Mert
4c7347d653
fix: re-add extends section for server in Compose files ( #9806 )
...
re-add extends section
2024-05-27 21:04:07 +00:00
Mert
dca420ef70
chore: refactor transcode config routing ( #9800 )
...
* chore: refactor transcode config
* rename parameter
* handle no /dev/dri
* prefer undefined
2024-05-27 15:20:07 -04:00
Mert
21bd20fd75
fix(server): nvenc not working when there are no filters ( #9802 )
...
don't add format=nv12
2024-05-27 15:18:01 -04:00
Mert
351dd647a9
feat(server): better video thumbnails ( #9784 )
2024-05-27 12:08:38 -04:00
Mert and Alex
e7c8501930
fix(server): search duplicates of the same asset type ( #9747 )
...
* search by type
* make sql
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-05-26 22:04:23 +00:00
Mert
69b5eb005f
fix(server): use qsv format for hwmap ( #9722 )
...
use qsv format for hwmap
2024-05-24 09:50:28 +01:00
Mert
a5e8b451b2
feat(server): qsv hardware decoding and tone-mapping ( #9689 )
...
* qsv hw decoding and tone-mapping
* fix vaapi
* add tests
* formatting
* handle device name without path
2024-05-23 03:58:29 +00:00
Mert
9aed736911
fix(ml): openvino not working with kernel 6.7.5 or later ( #9541 )
...
* add envs
* move to Dockerfile
2024-05-18 00:00:20 +00:00
Mert
ff52300624
refactor(server): duplicate controller and service ( #9542 )
...
* duplicate controller and service
* change endpoint name
* fix search tests
* remove unused import
* add to index
2024-05-16 19:39:33 -04:00
Mert
d8eca168ca
feat(server): fully accelerated nvenc ( #9452 )
...
* use arrayContaining
* libplacebo for nvenc
update dockerfile
* tweaks
* update nvenc options
* tweak settings
* refactor
* toggle for hardware decoding, software / hardware decoding for nvenc and rkmpp
* fix software tone-mapping not being applied
* separate configs for hw/sw
* update api
* add hw decode toggle
* fix mutating config
* remove `version` flag
* fix config type
* remove submodule
* handle temporal AQ
* remove duplicate tests
* use `tonemap_opencl`
* wording
* update docs
2024-05-16 13:30:26 -04:00
64636c0618
feat(server): near-duplicate detection ( #8228 )
...
* duplicate detection job, entity, config
* queueing
* job panel, update api
* use embedding in db instead of fetching
* disable concurrency
* only queue visible assets
* handle multiple duplicateIds
* update concurrent queue check
* add provider
* add web placeholder, server endpoint, migration, various fixes
* update sql
* select embedding by default
* rename variable
* simplify
* remove separate entity, handle re-running with different threshold, set default back to 0.02
* fix tests
* add tests
* add index to entity
* formatting
* update asset mock
* fix `upsertJobStatus` signature
* update sql
* formatting
* default to 0.03
* optimize clustering
* use asset's `duplicateId` if present
* update sql
* update tests
* expose admin setting
* refactor
* formatting
* skip if ml is disabled
* debug trash e2e
* remove from web
* remove from sidebar
* test if ml is disabled
* update sql
* separate duplicate detection from clip in config, disable by default for now
* fix doc
* lower minimum `maxDistance`
* update api
* Add and Use Duplicate Detection Feature Flag (#9364 )
* Add Duplicate Detection Flag
* Use Duplicate Detection Flag
* Attempt Fixes for Failing Checks
* lower minimum `maxDistance`
* fix tests
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com >
* chore: fixes and additions after rebase
* chore: update api (remove new Role enum)
* fix: left join smart search so getAll works without machine learning
* test: trash e2e go back to checking length of assets is zero
* chore: regen api after rebase
* test: fix tests after rebase
* redundant join
---------
Co-authored-by: Nicholas Flamy <30300649+NicholasFlamy@users.noreply.github.com >
Co-authored-by: Zack Pollard <zackpollard@ymail.com >
Co-authored-by: Zack Pollard <zack@futo.org >
2024-05-16 18:08:37 +01:00
Mert
ad915ccd64
docs(ml): update link ( #9477 )
...
update link
2024-05-14 15:34:36 +01:00
Mert
f667c9597b
chore(server): less rigid transcoding tests ( #9401 )
...
* use arrayContaining
* clean up
2024-05-12 00:04:31 -04:00
Mert
bb4843747b
perf: cache transcoding devices ( #9381 )
...
cache transcoding devices
2024-05-10 15:03:47 -04:00
Mert and Daniel Dietzler
f3fbb9b588
perf: cache getConfig ( #9377 )
...
* cache `getConfig`
* critical fix
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
---------
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
2024-05-10 18:15:25 +00:00
Mert and Alex
1167f0f2b7
feat(server): optimize person thumbnail generation ( #7513 )
...
* do crop and resize together
* redundant `pipelineColorspace` call
* formatting
* fix rebase
* handle orientation
* remove unused import
* formatting
* use oriented dimensions for half size calculation
* default case for orientation
* simplify orientation code
---------
Co-authored-by: Alex <alex.tran1502@gmail.com >
2024-05-08 08:09:34 -05:00
Mert
a0c43a2b5a
fix(server): person thumbnails not being queued during thumbnail generation ( #9195 )
...
use face id instead of asset id
2024-05-02 01:33:17 +01:00
Mert
2648032163
fix(server): untracked thumbnail and preview images ( #9126 )
...
* delete old thumbnails
* add tests
* revert invisible asset handling
2024-04-27 18:43:05 -04:00
Mert
d52ed51aab
fix(cli): dry run being inaccurate ( #9088 )
2024-04-25 21:14:48 -04:00
Mert
c14a2eda5d
fix(cli): --ignore getting ignored ( #9071 )
2024-04-25 10:48:17 -04:00
Mert
a90138e42e
docs(ml): clarify cuda and openvino requirements ( #9072 )
...
update docs
2024-04-24 23:15:26 -05:00
Mert
466451abc9
feat(server): env variable to skip migrations on startup ( #9069 )
...
* env variable to skip migrations
* update docs
* update env doc
2024-04-24 22:52:38 -04:00
Mert
c30cd3b378
chore: test more formats in e2e ( #9001 )
2024-04-22 01:35:27 -04:00
Mert
fd4514711f
feat(server): enable AV1 encoding for NVENC ( #8959 )
...
allow av1 for nvenc
2024-04-20 14:52:50 -04:00
Mert and Alex Tran
431ffebddd
feat(server): use embedded preview from raw images ( #8773 )
...
* extract embedded
* update api
* add tests
* move temp file logic outside of media repo
* formatting
* revert `toSorted`
* disable by default
* clarify setting description
* wording
* wording
* update docs
* check extracted image dimensions
* test that it unlinks
* formatting
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
2024-04-19 10:50:13 -05:00
Mert
596c35dc00
fix(server): skip invisible assets for thumbnail generation and ml ( #8891 )
...
* skip invisible assets for thumbnail generation and ml
* no need to update job status
* fix thumbhash check order
* linting
2024-04-19 01:37:55 +00:00
Mert
8573c84605
fix(server): include archived images in face detection ( #8892 )
2024-04-17 23:47:24 -04:00
Mert
8c9a092561
docs(ml): update hardware acceleration doc ( #8700 )
...
* update docs
* formatting
2024-04-11 09:39:18 +02:00
Mert
e13d4c9c13
chore: add code owner ( #8701 )
2024-04-11 06:34:19 +00:00
Mert
105a74caca
feat(server,web): configure image format ( #8581 )
2024-04-07 12:44:34 -04:00
Mert
55b9acca78
fix(server): hevc tag being set when copying a non-hevc stream ( #8582 )
2024-04-07 12:44:09 -04:00
Mert
0d130b8957
fix(server): x264/x265 params not being set correctly ( #8587 )
2024-04-07 12:43:50 -04:00
Mert
0aa5d3daeb
fix(web): reset to default button always being shown ( #8577 )
2024-04-07 12:43:40 -04:00
Mert
335c03d0b8
chore(server): better typing for system config key ( #8580 )
...
* config type safety
* typeorm fix
* typing fixes
* don't use enum in db
* add todo
2024-04-07 01:47:33 +00:00
Mert
4681ff88d0
fix(server): image config not being updated ( #8579 )
...
update system config key
2024-04-06 21:06:26 -04:00
Mert
33fd27f113
fix(web): some settings not disabled when using config file ( #8576 )
...
add disable
2024-04-06 20:59:00 -04:00
Mert and Alex Tran
8edc2fb46f
refactor(server): decouple generated images from image formats ( #8246 )
...
* rename
thumbnail config
update target paths, fix tests
rename to image settings
replace legacy enum
better typing
update sql
update api
remove config option
fix
* update docs
* update other thumbnail configs in migration
* keep legacy enum for now
* fix jumbled job names
* fix jumbled job names in tests
* rename thumbhash job
* rename dto
* fix tests
* preserve order
* remove unused import
* keep old fields in dto, marked deprecated
* update sql
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
2024-04-02 04:56:56 +00:00
Mert
169d9d18b0
docs: document metric env variables, add job metric env ( #8406 )
...
* update env docs
* show options
2024-03-31 17:29:11 +00:00
Mert
5bc9158724
fix(server): penalize null geodata fields when searching places ( #8408 )
2024-03-31 10:59:11 -04:00
Mert
395c28f5fa
fix(server): parameter for all places query ( #8346 )
...
* fix parameter
* add e2e
2024-03-31 02:29:02 +00:00
Mert
c45e28ab53
refactor(server): metric repo ( #8278 )
...
* refactor
* redundant `implements`
* simplify
* remove `enabled`
2024-03-25 23:15:11 +00:00
Mert
4a6a0aa142
fix(server): places page not working with partner sharing ( #8257 )
2024-03-25 01:59:11 -04:00
Mert
c58a70ac8f
feat(server): job metrics ( #8255 )
...
* metric repo
* add metric repo
* remove unused import
* formatting
* fix
* try disabling job metrics for e2e
* import otel in test module
2024-03-24 23:02:04 -04:00
Mert
b07a565e34
chore(server): change upsert signature for search repo ( #8210 )
...
* upsert embedding
* remove unused imports
2024-03-23 14:37:06 -04:00
Mert
604b8ff17c
chore(server): remove getByDate from asset repo ( #8211 )
...
* remove getByDate
* remove unused import
2024-03-22 23:20:16 -05:00
c4e910dd3d
docs(server): add documentation for prometheus metrics ( #8084 )
...
* add monitoring doc
* wording
* indent
* note instead of tip
* Update docs/docs/features/monitoring.md
Co-authored-by: bo0tzz <git@bo0tzz.me >
* Update docs/docs/features/monitoring.md
Co-authored-by: bo0tzz <git@bo0tzz.me >
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
Co-authored-by: bo0tzz <git@bo0tzz.me >
2024-03-20 09:20:46 -05:00
Mert
f392fe7702
fix(server): "view all" for cities only showing 12 cities ( #8035 )
...
* view all cities
* increase limit
* rename endpoint
* optimize query
* remove pagination
* update sql
* linting
* revert sort by count in explore page for now
* fix query
* fix
* update sql
* move to search, add partner support
* update sql
* pr feedback
* euphemism
* parameters as separate variable
* move comment
* update sql
* linting
2024-03-20 03:23:57 +00:00
Mert
2daed747cd
chore(server): change save -> update in asset repository ( #8055 )
...
* `save` -> `update`
* change return type
* include relations
* fix tests
* remove when mocks
* fix
* stricter typing
* simpler type
2024-03-19 22:42:10 -04:00
Mert
3a045b33ca
chore(deps): update onnxruntime-openvino ( #7854 )
2024-03-16 00:04:45 -04:00
Mert and Alex Tran
a9438a9c2d
fix(server): prevent feedback loop during library scan ( #7944 )
...
* prevent feedback loop
* add nesting
* made nesting less ugly
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com >
2024-03-15 22:01:58 +00:00
Mert
ee8e8a0c0f
perf(server): optimize getByIds query ( #7918 )
...
* clean up usage
* i'm not updating all these tests
* update tests
* add indices
* add indices to entities
remove index from person entity
add to face entity
fix
* simplify query
* update sql
* missing await
* remove synchronize false
2024-03-14 01:58:09 -04:00
Mert
d67cc00e4e
feat(server): lower library scan memory usage ( #7939 )
...
* use trie
* update tests
* formatting
* pr feedback
* linting
2024-03-14 01:52:30 -04:00
a097e903c9
feat(server): OpenTelemetry integration ( #7356 )
...
* wip
* span class decorator
fix typing
* improvements
* noisy postgres logs
formatting
* add source
* strict string comparison
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
* remove debug code
* execution time histogram
* remove prometheus stuff
remove prometheus data
* disable by default
disable nestjs-otel stuff by default
update imports
* re-add postgres instrumentation
formatting
formatting
* refactor: execution time histogram
* decorator alias
* formatting
* keep original method order in filesystem repo
* linting
* enable otel sdk in e2e
* actually enable otel sdk in e2e
* share exclude paths
* formatting
* fix rebase
* more buckets
* add example setup
* add envs
fix
actual fix
* linting
* update comments
* update docker env
* use more specific env
---------
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com >
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com >
2024-03-12 05:19:12 +00:00
Mert
d09980f646
chore(cli): clarify use of concurrency option ( #7840 )
...
* add <number>
* add e2e tests
* add test with number
2024-03-11 23:30:13 -04:00
Mert
8dc62bd29a
fix(server): face search results not always sorted ( #7839 )
...
* order by
* update sql
2024-03-11 09:19:20 -05:00
Mert
5bd597f14b
fix(server): external library sync not working for large libraries ( #7759 )
2024-03-10 22:30:57 -04:00
Mert
1ec5d612fa
perf(server): use queries to refresh library assets ( #7685 )
...
* use queries instead of js
* missing await
* add mock methods
* fix test
* update sql
* linting
2024-03-06 21:23:10 -06:00
Mert
ffaa08e7ea
chore(server): lower default max recognition distance for facial recognition ( #7689 )
...
lower default to 0.5
2024-03-06 22:20:38 -05:00
Mert
f88343019d
perf(web): optimize response sizes for initial page load ( #7594 )
2024-03-06 12:05:53 -05:00
Mert
762c4684f8
perf(web): optimize date groups ( #7593 )
...
* optimize date groups
* remove `.values()`
* remove console.log
* remove if condition
* remove console.log
* remove outdated comment
* revert dynamic import
2024-03-03 16:12:52 -06:00
Mert
2fa10a254c
feat(web): improve alt text ( #7596 )
...
* alt text
* memory lane alt text
* revert sql generator change
* use getAltText
* oops
* handle large number of people in asset
* nit
* add aria-label to search button
* update api
* fixed tests
* fixed typing
* fixed spacing
* fix displaying null
2024-03-03 16:42:17 -05:00
Mert
8b02f18e99
fix(web): prevent duplicate calls to time bucket endpoint ( #7563 )
2024-03-01 14:16:07 -05:00
Mert
dc7a329cc8
fix(server): only queue ml / transcoding jobs after thumbnail generation on upload ( #7516 )
2024-02-28 23:23:21 -05:00
Mert
