Commit Graph
77 Commits
Author SHA1 Message Date
Jonathan Jogenfors b411e30796 fix(server): only allow absolute import paths (#13642)
fix: only allow absolute paths
2024-10-21 09:12:12 -05:00
Jonathan Jogenfors 0b48d46402 chore: Add viteconfig-timestamp to .gitignore (#13380) 2024-10-11 20:51:46 +02:00
Jonathan Jogenfors b59abdff3d chore(e2e): dont check for immich folder (#13298)
chore: dont check immich folder
2024-10-10 15:42:59 +07:00
Jonathan Jogenfors b7dcc97712 fix(cli): allow special characters in paths (#13282)
* fix(cli): commas in import paths

* adding more test cases
2024-10-09 18:15:50 +02:00
Jonathan Jogenfors 5b00bc499f fix(server): Allow commas and braces in import paths (#13259)
fix commas and braces in paths
2024-10-07 15:43:21 -04:00
b2f2be3485 refactor(server): library syncing (#12220)
* refactor: library scanning

fix tests

remove offline files step

cleanup library service

improve tests

cleanup tests

add db migration

fix e2e

cleanup openapi

fix tests

fix tests

update docs

update docs

update mobile code

fix formatting

don't remove assets from library with invalid import path

use trash for offline files

add migration

simplify scan endpoint

cleanup library panel

fix library tests

e2e lint

fix e2e

trash e2e

fix lint

add asset trash tests

add more tests

ensure thumbs are generated

cleanup svelte

cleanup queue names

fix tests

fix lint

add warning due to trash

fix trash tests

fix lint

fix tests

Admin message for offline asset

fix comments

Update web/src/lib/components/asset-viewer/asset-viewer-nav-bar.svelte

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

add permission to library scan endpoint

revert asset interface sort

add trash reason to shared link stub

improve path view in offline

update docs

improve trash performance

fix comments

remove stray comment

* refactor: add back isOffline and remove trashReason from asset, change sync job flow

* chore(server): drop coverage to 80% for functions

* chore: rebase and generated files

---------

Co-authored-by: Zack Pollard <zackpollard@ymail.com>
2024-09-25 18:26:19 +01:00
Jonathan Jogenfors 233372303b feat(server): default exclusion patterns (#12566)
* Add default exclusion patterns

* simplify

* fix tests
2024-09-11 09:40:52 -05:00
Jonathan Jogenfors 39141d3f1c fix(server): remove offline assets from trash (#12199)
* use port not taken by immich-dev for e2e

* remove offline files from trash
2024-09-02 01:06:35 +02:00
Jonathan Jogenfors aa04ded311 chore(e2e): change e2e ports to some not used by immich-dev (#12132)
use port not taken by immich-dev for e2e
2024-08-30 08:04:02 -04:00
Jonathan JogenforsandJason Rasmussen eb4a291c81 chore(server): log path when generating external thumbnail (#12107)
* feat: log path when generating external thumbnail

* chore: clean up

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-08-29 22:16:12 +00:00
Jonathan Jogenfors bab5ad7ebd fix(server): ensure new exclusion patterns work (#12102)
* add test for bug

* find excluded paths when checking offline

* fix filename

* fix unit tests

* bump picomatch

* fix e2e paths

* improve e2e

* add unit tests

* cleanup e2e

* set correct asset count

* fix e2e test

* fix lint
2024-08-28 19:51:25 -04:00
363c558db7 fix(server): don't crash when refreshing large libraries (#7934)
* add job to check for offline files

* fix lint

* only check for offline when using checkForOffline

* improve tests

* remove old test

* wip

* remove trie

* refactor batches

* also check offline status

* fix spelling

* don't do offline scan

* rename scan to check

* fix job statuses

* fix lint

* cleanup

* add test

* open-api

* fix test

* fix spinner

* reset text

* don't double batch

* fix comments from mert

* remove tries

* fix tests

* fix e2e

* fix test

* fix test

* add tests

* fix lint

* fix e2e

* interweave scans

* fix errors

* fix messages

* fix test

* add mock

* fix sql

* fix e2e

* use library batch size

* save -> update

* add file extensions

* update specs

* test for import paths

* check import paths when testing offline

* fix lint

* normalize import path

* remove console logs

* decrease batch size to 1000

* add test for import path

* add test for already-online assets

* fix merge

* fix lint

* add library job back

* add offline job to correct queue

* library spec compiles now

* move one test to new e2e

* fix comments

* fix comments

* fix lint

* refactor path validation

* fix loop bug

* remove logging

* expect responses

* fix asset mock

* take the straightforward approach

* use generator correctly

* fix vitest on file edit

* bump vitest to 1.6.0

* test for offline check

* add e2e tests for offlining assets depending on import path

* cleanup e2e test after finish

* cleanup library service

* paginate the walk generator

* fix tests

* fix typo

* refactoring handleOfflineCheck

* better testing of handleOfflineCheck

* fix lint

* handle large library deletions

* dont check if library is deleted

* fix mock

* add a 100k page size to library

* fix loading animation

* better log messages

* Better logging for offline asset removal

* fix sql and tests

* fix number format

* Remove submodule

* fix format

* chore: cleanup

* chore: fix tests

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-08-28 13:05:48 -04:00
Jonathan Jogenfors bc8e236598 chore(server): make vite-tsconfig-paths a dev dependency instead (#11404) 2024-07-27 21:50:35 +02:00
Jonathan Jogenfors 0a6e5e0ec1 fix(server): make vitest pick up edited files (#11385)
fix vitest on file edit
2024-07-26 16:26:38 -05:00
Jonathan Jogenfors 65a4f86154 chore: bump vitest to 1.6.0 (#11386)
bump vitest to 1.6.0
2024-07-26 16:26:17 -05:00
Jonathan Jogenfors ec48fccb30 fix(server): add missing file extensions to library files (#8342)
* fix file extensions

* fix tests

* fix formatting

* fixed bug

* fix merts comments
2024-03-28 22:51:07 -04:00
Jonathan JogenforsandAlex Tran e6f2bb9f89 fix(server): use extension in originalFileName for libraries (#8083)
* use file base

* fix: test

* fix: e2e-job tests

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-19 23:40:28 -05:00
Jonathan Jogenfors ba38713fbc fix(server): queue library asset refresh in batches (#7914)
* add debug logs

* scan assets in batches

* Cleanup

* don't normalize

* Removing extra log

* remove unneeded code

* change log levels
2024-03-14 14:43:05 +01:00
Jonathan Jogenfors a5a27594b8 docs: admin-only library settings (#7716)
* document admin-only library settings

* renew library guide
2024-03-07 21:27:02 +00:00
Jonathan Jogenfors f1a8e385e9 deps(server): CVE-2024-28176 (#7717)
bumpety bump
2024-03-07 15:34:10 -05:00
4cb0f37918 chore(server): Move library watcher to microservices (#7533)
* move watcher init to micro

* document watcher recovery

* chore: fix lint

* add try lock

* use global library watch lock

* fix: ensure lock stays on

* fix: mocks

* unit test for library watch lock

* move statement to correct test

* fix: correct return type of try lock

* fix: tests

* add library teardown

* add chokidar error handler

* make event strings an enum

* wait for event refactor

* refactor event type mocks

* expect correct error

* don't release lock in teardown

* chore: lint

* use enum

* fix mock

* fix lint

* fix watcher await

* remove await

* simplify typing

* remove async

* Revert "remove async"

This reverts commit 84ab5abac4.

* can now change watch settings at runtime

* fix lint

* only watch libraries if enabled

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-07 11:36:53 -06:00
Jonathan Jogenfors ae46188753 chore(deps): bump sanitize-html, fixing CVE-2024-21501 (#7662)
* bump sanitize-html

* bump better
2024-03-05 17:35:52 -05:00
5d377e5b0f chore(server): eslint await-thenable (#7545)
* await-thenable

* fix library watchers

* moar eslint

* fix test

* fix typo

* try to remove check void return

* fix checksVoidReturn

* move to domain utils

* remove eslint ignores

* chore: cleanup types

* chore: use logger

* fix: e2e

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-05 17:23:06 -05:00
efa6efd200 feat(server,web): remove external path nonsense and make libraries admin-only (#7237)
* remove external path

* open-api

* make sql

* move library settings to admin panel

* Add documentation

* show external libraries only

* fix library list

* make user library settings look good

* fix test

* fix tests

* fix tests

* can pick user for library

* fix tests

* fix e2e

* chore: make sql

* Use unauth exception

* delete user library list

* cleanup

* fix e2e

* fix await lint

* chore: remove unused code

* chore: cleanup

* revert docs

* fix: is admin stuff

* table alignment

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-29 12:35:37 -06:00
Jonathan Jogenfors e4f32a045d chore: remove watcher polling option (#7480)
* remove watcher polling

* fix lint

* add db migration
2024-02-28 21:20:10 +01:00
Jonathan Jogenfors 0d2a656aa1 chore: add agpl milestone (#7479)
add agpl milestone
2024-02-27 21:43:12 +00:00
Jonathan JogenforsandAlex Tran b3c7bebbd4 feat(server,web) Semantic import path validation (#7076)
* add library validation api

* chore: open api

* show warning i UI

* add flex row

* fix e2e

* tests

* fix tests

* enforce path validation

* enforce validation on refresh

* return 400 on bad import path

* add limits to import paths

* set response code to 200

* fix e2e

* fix lint

* fix test

* restore e2e folder

* fix import

* use startsWith

* icon color

* notify user of failed validation

* add parent div to validation

* add docs to the import validation

* improve library troubleshooting docs

* fix button alignment

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-20 09:53:12 -06:00
Jonathan Jogenforsandbo0tzz 6e853e2a9d fix(cli): Run npm ci in sdk folder before deploying to npm (#6986)
* fix cli path

* run npm ci in sdk folder

* version bump

* build sdk before publishing cli

* dry run publish

* build cli

* fix(ci): Push CLI latest tag

* remove dry run flag

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-02-11 23:41:29 +01:00
Jonathan Jogenfors 954c1c2ef4 feat(server): server-side checking of duplicate import paths and exclusion patterns (#6993)
validate path and pattern
2024-02-09 13:02:11 -05:00
Jonathan Jogenfors b67fddf4b8 fix(web): Handle duplicate library settings gracefully (#6950)
* don't add duplicate import paths

* improve library import paths form

* same for exclusion patterns

* remove newline
2024-02-09 00:09:09 +00:00
Jonathan Jogenforsandbo0tzz 36477d59c4 chore(cli): publish docker image on ghcr (#6964)
* build(cli): Docker release workflow

* build(cli): Run jobs for main/PR pushes

* chore(ci): Rename cli build workflow

* chore(ci): Fix CLI build event paths

* chore(ci): CLI build cleanup

* build(cli): Fix build-push-action dockerfile ref

* chore: Remove redundant comment

* build(cli): Fix registry login step

* build(cli): Manually define latest tag

* build(cli): Only push on workflow_dispatch

* build(cli): Run docker release after NPM publish

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-02-08 11:26:24 -05:00
Jonathan JogenforsandBen McCann 31eb4790dc feat(cli): dockerize (#6858)
* import dockerfile from old cli

* build works

* rename login command

* bump packages

* fix login command

* chore: remove axios dependency from CLI

* move immich script path

* can build docker

* wip

* wip

* don't externalize sdk

* can run docker

* improve entrypoint

* can save auth state between runs

* add docs

* clarify reqs

* fix lint

* bump alpine to 3.19

* add env files for api key

* remove immich cli GHA for now

* Update docs/docs/features/command-line-interface.md

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

* remove redundant env variable check

* cleanup

* speling

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
2024-02-06 12:17:15 +01:00
Jonathan Jogenfors 755444e9a4 chore(cli): use upload api and update documentation (#6927)
* use fetch api

* bump version

* add documentation

* revert to using file blob
2024-02-06 11:00:35 +01:00
Jonathan Jogenfors ce6dc3b7af fix(cli): auth file should be chmod 600 (#6925)
* wip new tests

* test for auth file mode

* check perms internally

* chore: lint
2024-02-06 00:40:22 +01:00
f44fa45aa0 chore(server,cli,web): housekeeping and stricter code style (#6751)
* add unicorn to eslint

* fix lint errors for cli

* fix merge

* fix album name extraction

* Update cli/src/commands/upload.command.ts

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

* es2k23

* use lowercase os

* return undefined album name

* fix bug in asset response dto

* auto fix issues

* fix server code style

* es2022 and formatting

* fix compilation error

* fix test

* fix config load

* fix last lint errors

* set string type

* bump ts

* start work on web

* web formatting

* Fix UUIDParamDto as UUIDParamDto

* fix library service lint

* fix web errors

* fix errors

* formatting

* wip

* lints fixed

* web can now start

* alphabetical package json

* rename error

* chore: clean up

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-01 22:18:00 -05:00
Jonathan Jogenforsandmartabal c083636c51 docs: update milestones (#6787)
* fix zero index

* fix day

* docs: add gpu acceleration milestone

---------

Co-authored-by: martabal <74269598+martabal@users.noreply.github.com>
2024-01-31 07:01:53 -05:00
068e703e88 feat(server): Automatic watching of library folders (#6192)
* feat: initial watch support

* allow offline files

* chore: ignore query errors when resetting e2e db

* revert db query

* add savepoint

* guard the user query

* chore: openapi and db migration

* wip

* support multiple libraries

* fix tests

* wip

* can now cleanup chokidar watchers

* fix unit tests

* add library watch queue

* add missing init from merge

* wip

* can now filter file extensions

* remove watch api from non job client

* Fix e2e test

* watch library with updated import path and exclusion pattern

* add library watch frontend ui

* case sensitive watching extensions

* can auto watch libraries

* move watcher e2e tests to separate file

* don't watch libraries from a queue

* use event emitters

* shorten e2e test timeout

* refactor chokidar code to filesystem provider

* expose chokidar parameters to config file

* fix storage mock

* set default config for library watching

* add fs provider mocks

* cleanup

* add more unit tests for watcher

* chore: fix format + sql

* add more tests

* move unwatch feature back to library service

* add file event unit tests

* chore: formatting

* add documentation

* fix e2e tests

* chore: fix e2e tests

* fix library updating

* test cleanup

* fix typo

* cleanup

* fixing as per pr comments

* reduce library watch config file

* update storage config and mocks

* move negative event tests to unit tests

* fix library watcher e2e

* make watch configuration global

* remove the feature flag

* refactor watcher teardown

* fix microservices init

* centralize asset scan job queue

* improve docs

* add more tests

* chore: open api

* initialize app service

* fix docs

* fix library watch feature flag

* Update docs/docs/features/libraries.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* fix: import right app service

* don't be truthy

* fix test speling

* stricter library update tests

* move fs watcher mock to external file

* subscribe to config changes

* docker does not need polling

* make library watch() private

* feat: add configuration ui

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 08:15:54 +00:00
Jonathan Jogenfors f6afb23d56 docs(server): Fix glob pattern in library exclusions (#6752)
fix glob pattern in library exclusions
2024-01-30 07:57:22 -05:00
Jonathan Jogenfors 33757689fe fix(deps) bump msgpackr (#6673)
bump msgpackr
2024-01-26 16:41:12 +00:00
Jonathan Jogenfors 78a2fa8569 fix(deps): bump lodash (#6672)
bump lodash
2024-01-26 11:39:47 -05:00
Jonathan Jogenfors 0d3929f732 fix(deps) bump marked to 11.1.1 (#6670)
bump marked to 11.1.1
2024-01-26 11:32:01 -05:00
Jonathan Jogenfors 2d0db5d0f2 fix(deps) CVE-2022-25883 (#6667)
fix CVE-2022-25883
2024-01-26 09:24:35 -06:00
Jonathan Jogenfors ee82327d2a add tests for cli exclusion pattern (#6148) 2024-01-03 17:42:26 -05:00
8295542941 feat(cli): Add existing assets to album and allow album name (#5838)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* feat: always skip hashing when adding albums

* feat: create album with specific name

* check asset duplication before adding to album

* update documentation

* use correct check for when to create albums

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-20 09:51:53 -06:00
5a3a2c7293 feat(cli): Allow uploading a single file (#5837)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* feat: can upload single file

* fix upload options dto

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-19 13:15:11 -06:00
f4c5bdfa1c fix(cli): don't open too many files (#5841)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* chore: remove old gha step

* add npm ci to server

* feat: use graceful-fs

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-19 09:40:29 -06:00
4e9b96ff1a test(cli): e2e testing (#5101)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* chore: remove old gha step

* add npm ci to server

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-18 20:29:26 -06:00
Jonathan Jogenfors 81259115d1 chore(cli): set cli workdir in npm publish (#5185)
* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url
2023-11-20 13:36:17 +00:00
Jonathan Jogenfors c1239a7337 docs: remove old CLI first (#5163) 2023-11-19 16:41:42 -06:00
Jonathan JogenforsandAlex 7e38e7c949 feat(cli): refactor and add album support (#4434)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* chore: remove unneeded packages

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-11-19 22:16:24 +00:00
Jonathan Jogenfors 7a8f8e5472 chore: don't set max old space (#5050) 2023-11-14 16:32:58 -06:00
Jonathan JogenforsandAlex Tran dadcf49eca fix(server,web): correctly remove metadata from shared links (#4464)
* wip: strip metadata

* fix: authenticate time buckets

* hide detail panel

* fix tests

* fix lint

* add e2e tests

* chore: open api

* fix web compilation error

* feat: test with asset with gps position

* fix: only import fs.promises.cp

* fix: cleanup mapasset

* fix: format

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-14 01:46:30 +00:00
Jonathan Jogenfors f0bb50b61a fix(server,cli): don't float promises (#4433)
* fix: don't allow floating promises

* fix: await all promises

* fix: download archives

* fix cli tests

* fix: skip web
2023-10-13 01:22:40 -04:00
Jonathan JogenforsandAlex Tran 41befc0948 fix(server): don't publicly reveal user count (#4409)
* fix: don't reveal user count publicly

* fix: mobile and user controller

* fix: update other frontend endpoints

* fix: revert openapi change

* chore: open api

* fix: initialize

* openapi

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-11 02:37:13 +00:00
Jonathan JogenforsandJason Rasmussen 56eb7bf0fc fix(server): improve library scan queuing performance (#4418)
* fix: inline mark asset as offline

* fix: improve log message

* chore: lint

* fix: offline asset algorithm

* fix: use set comparison to check what to import

* fix: only mark new offline files as offline

* fix: compare the correct array

* fix: set default library concurrency to 5

* fix: remove one db call when scanning new files

* chore: remove unused import

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-10 18:59:13 -04:00
Jonathan Jogenfors 83b63ca12e fix: only check external path once (#4419) 2023-10-10 13:44:43 +00:00
Jonathan Jogenfors f57acc0802 fix(server): add original path and library id index to asset (#4410)
* fix: add original path index

* fix: use libraryId for index, too

* fix: revert openapi change
2023-10-10 09:38:26 -04:00
Jonathan Jogenfors b56f22aac3 docs: add troubleshooting info to libraries + minor docs tweaks (#4377)
* docs: add troubleshooting info to libraries

* fix: revert docker-compose
2023-10-09 09:13:47 -05:00
Jonathan Jogenfors 4dffae3f39 fix(server): normalize external path (#4239)
* fix: use normalized external path

* fix: move normalization to user core
2023-10-06 20:47:38 +00:00
Jonathan Jogenfors 81009c17bf fix(server): use fileCreatedAt in buckets (#4354) 2023-10-05 08:26:42 -05:00
Jonathan Jogenfors e2f1e38472 chore(server,web): bump node version to 20.8 (#4311)
* chore: bump node version to 20.8

* fix: remove node hash
2023-10-03 09:34:35 -05:00
Jonathan Jogenfors 6d1868a6e0 feat: server containers use host timezone (#4313) 2023-10-02 20:50:27 -05:00
Jonathan JogenforsandJason Rasmussen 3172c341e0 chore(server): bump exiftool-vendored to 23.1.0 (#4302)
* chore: bump exiftool-vendored

* fix: correct version

* chore: bump exiftool-vendored.pl

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-02 08:54:43 -04:00
Jonathan JogenforsandJason Rasmussen 8873c9a02f docs: deprecate read only assets (#4226)
* chore: add deprecation notice in docs

* feat: link to library documentation

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-26 02:26:11 +00:00
Jonathan Jogenfors 9676412875 fix: cli import (#4224)
* fix: allow import of assets

* fix: allow deletion of read only assets
2023-09-25 19:04:30 -04:00
acdc66413c feat(server,web): libraries (#3124)
* feat: libraries

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-09-20 13:16:33 +02:00
Jonathan Jogenfors 3edade6761 feat(server): tsconfig: don't clear console on nest startup (#4032)
* feat: don't clear nest output

* feat: moved config to tsconfig

* fix: cleanup package json
2023-09-08 11:17:45 -04:00
Jonathan JogenforsandJason Rasmussen efcc66d63b feat(web): only log http errors in web container (#4031)
* feat: reduce web container log verbosity on error

* fix: web test

* feat: better error logging

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-08 15:09:59 +00:00
Jonathan Jogenfors a404fb6cb5 feat: don't clear nest output (#4023) 2023-09-08 14:45:00 +07:00
Jonathan Jogenfors ea64fdd7b4 feat(server): Use Testcontainers for e2e tests (#3202)
* Add testcontainers to e2e

* Run e2e tests in github without docker

* Run lint on e2e

* Cleaner setup ordering

* Rename setup file
2023-07-11 16:54:44 -05:00
Jonathan JogenforsandAlex Tran 6f4449d5e9 feat(cli) Add new CLI (#3066)
* Add new cli

* Remove old readme

* Add documentation to readme file

* Add github workflow tests for cli

* Fix typo in docs

* Add usage info to readme

* Add package-lock.json

* Fix tsconfig

* Cleanup

* Fix lint

* Cleanup package.json

* Fix accidental server change

* Remove rootdir from cli

* Remove tsbuildinfo

* Add prettierignore

* Make CLI use internal openapi specs

* Add ignore and dry-run features

* Sort paths alphabetically

* Don't remove substring

* Remove shorthand for delete

* Remove unused import

* Remove chokidar

* Set correct openapi cli generator script

* Add progress bar

* Rename target to asset

* Add deletion progress bar

* Ignore require statement

* Use read streams instead of readfile

* Fix github feedback

* Fix upload requires

* More github comments

* Cleanup messages

* Cleaner pattern concats

* Github comments

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-07-06 09:37:47 -05:00
Jonathan Jogenfors f003ff3c98 Add dependency on immich-web to immich-proxy (#2875) 2023-06-20 08:46:48 -05:00
1b54c4f8e7 feat(server): Add support for client-side hashing (#2072)
* Modify controller DTOs

* Can check duplicates on server side

* Remove deviceassetid and deviceid

* Remove device ids from file uploader

* Add db migration for removed device ids

* Don't sanitize checksum

* Convert asset checksum to string

* Make checksum not optional for asset

* Use enums when rejecting duplicates

* Cleanup

* Return of the device id, but optional

* Don't use deviceId for upload folder

* Use checksum in thumb path

* Only use asset id in thumb path

* Openapi generation

* Put deviceAssetId back in asset response dto

* Add missing checksum in test fixture

* Add another missing checksum in test fixture

* Cleanup asset repository

* Add back previous /exists endpoint

* Require checksum to not be null

* Correctly set deviceId in db

* Remove index

* Fix compilation errors

* Make device id nullabel in asset response dto

* Reduce PR scope

* Revert asset service

* Reorder imports

* Reorder imports

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Update openapi

* Reduce PR scope

* refactor: asset bulk upload check

* chore: regenreate open-api

* chore: fix tests

* chore: tests

* update migrations and regenerate api

* Feat: use checksum in web file uploader

* Change to wasm-crypto

* Use crypto api for checksumming in web uploader

* Minor cleanup of file upload

* feat(web): pause and resume jobs

* Make device asset id not nullable again

* Cleanup

* Device id not nullable in response dto

* Update API specs

* Bump api specs

* Remove old TODO comment

* Remove NOT NULL constraint on checksum index

* Fix requested pubspec changes

* Remove unneeded import

* Update server/apps/immich/src/api-v1/asset/asset.service.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove unneeded check

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove hashing in the web uploader

* Cleanup file uploader

* Remove varchar from asset entity fields

* Return 200 from bulk upload check

* Put device asset id back into asset repository

* Merge migrations

* Revert pubspec lock

* Update openapi specs

* Merge upstream changes

* Fix failing asset service tests

* Fix formatting issue

* Cleanup migrations

* Remove newline from pubspec

* Revert newline

* Checkout main version

* Revert again

* Only return AssetCheck

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
2023-05-24 16:08:21 -05:00
Jonathan Jogenfors d2600e0ddd Documentation: Add FAQ items for reverse proxy and rpi performance (#2070)
* Add FAQ items for reverse proxy and rpi performance

* Fix wording
2023-03-24 09:36:44 -05:00
Jonathan Jogenfors d7bfab7b13 Document cli path parameter (#2011) 2023-03-18 22:11:02 +01:00
Jonathan Jogenfors 6c8b29f326 Document fallback timezone setting (fixes #2000) (#2003)
* Update FAQ.md

* Fix typo

* Mention the extract metadata job

* Don't duplicate code
2023-03-16 12:19:35 -05:00
Jonathan Jogenfors 54831878e0 Fix storage template extension display (#2002)
* Display correct jpg file extension

* Fix typo in template directory

* Move storage template to correct spelling
2023-03-15 14:39:29 -05:00