Skip to content
Merged

V6.0 #342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
b7cd491
add asynq
Dec 12, 2024
e9c7827
add tasks, connect
Dec 12, 2024
cb4fb48
setup redis config
Dec 12, 2024
65dabae
fix format
Dec 12, 2024
238f055
add worker
Dec 12, 2024
66e2721
update gitignore
Jan 1, 2025
9b19bc7
update
Jan 1, 2025
87d9ed0
run http server and asynq worker in separate goroutines
Jan 1, 2025
f90f1aa
get a task running
Jan 1, 2025
cfcc8ad
Merge pull request #324 from Receipt-Wrangler/task/add-asynq-shell
Noah231515 Jan 2, 2025
b046991
move quick scan to use asynq queue
Jan 2, 2025
76760b9
update swagger
Jan 2, 2025
9c486c7
add air
Jan 3, 2025
9104bd4
add logging
Jan 3, 2025
2829df5
refactor quick scan to pick up file from tmp directory, and delete it…
Jan 3, 2025
d450853
switch to crypto rand
Jan 3, 2025
d33f460
Merge pull request #326 from Receipt-Wrangler/feature/quick-scan-async
Noah231515 Jan 3, 2025
8af3af9
start up and shutdown scheduler
Jan 4, 2025
a95fab4
move router construction to separate package
Jan 4, 2025
7044a26
move things around and get email poll set up
Jan 5, 2025
dd88637
clean up circular dependencies
Jan 6, 2025
58ffff6
clean up left over broken imports
Jan 6, 2025
f919566
set up process handler
Jan 6, 2025
e918481
enqueue tasks instead of do tasks
Jan 6, 2025
b4a0b88
fix errors, and start
Jan 6, 2025
45a7a9b
implement file cleanup
Jan 6, 2025
f8f86e5
call cleanup
Jan 7, 2025
1081829
remove asynq task idon system settings
Jan 7, 2025
ffa2944
fix delete
Jan 7, 2025
a374b0f
update log to add in where it was called from
Jan 7, 2025
e800a49
update default prompt
Jan 7, 2025
d1c2012
don't retry polling
Jan 7, 2025
811ebff
pass in max retry
Jan 7, 2025
5cb894c
only attempt to delete if it exists
Jan 7, 2025
7652053
reminder
Jan 8, 2025
4676a41
fix test
Jan 8, 2025
3431bd6
add test
Jan 8, 2025
6bd703a
add comments
Jan 8, 2025
98846d7
adjust comments
Jan 9, 2025
2a7635f
Merge pull request #330 from Receipt-Wrangler/feature/email-asynq
Noah231515 Jan 9, 2025
b6433f1
move envs to constants
Jan 9, 2025
bd94fcb
add user/password support
Jan 9, 2025
9f538bf
Merge pull request #332 from Receipt-Wrangler/enhancement/add-user-pa…
Noah231515 Jan 9, 2025
9729ac3
update models
Jan 9, 2025
aec682b
add queue priorities
Jan 9, 2025
be8dd1c
config instead
Jan 9, 2025
8f03236
split queue names into separate file
Jan 9, 2025
9eb789e
remove simpleutils package
Jan 9, 2025
3c55364
clean up references, and constant variable names
Jan 9, 2025
8725b7a
move queue names into models
Jan 9, 2025
9405002
clean up other bad references, fix env package name
Jan 9, 2025
1e49ccc
add queue config to system settings
Jan 9, 2025
66ecddc
update swagger
Jan 9, 2025
af4a41b
udpate startup to get default configs
Jan 10, 2025
9785583
simplify system settings updating
Jan 10, 2025
8f4ac9a
update swagger
Jan 10, 2025
ac8c3c7
fix update statement
Jan 10, 2025
62a7e79
merge with main
Jan 10, 2025
402806c
update group handler
Jan 10, 2025
40900fa
get default asynq queue configurations if none
Jan 10, 2025
012af6a
update configs
Jan 10, 2025
a6e6623
fix tests
Jan 14, 2025
9c7e96e
implement restart
Jan 14, 2025
8737e2c
add some logging
Jan 14, 2025
31b5ab6
convert asynq to task
Jan 14, 2025
a9758a3
Merge pull request #334 from Receipt-Wrangler/feature/add-asynq-setti…
Noah231515 Jan 14, 2025
ba4ef2b
update widget type
Jan 15, 2025
30e2eee
update widget type
Jan 15, 2025
c3f9da3
add activity struct
Jan 15, 2025
dfce1ed
add can be restarted
Jan 15, 2025
89b7812
add activity to paged data
Jan 15, 2025
c2122b9
add request command
Jan 15, 2025
ffb9ecd
add endpoint and implement data fetching
Jan 15, 2025
d5b1821
fix dashboards getting created with empty widget configuration
Jan 16, 2025
68b8b89
fix swagger
Jan 16, 2025
ded8820
remove child tasks
Jan 16, 2025
a088b65
fix query, move filtered tasks out to variable
Jan 16, 2025
03bed46
cleanup
Jan 16, 2025
9b959b8
adding missing receipt_updated
Jan 18, 2025
f84747c
create RECEIPT_UPLOADED task on create
Jan 18, 2025
0c12507
update swagger
Jan 18, 2025
6ee8fe5
remove field
Jan 18, 2025
ccf193a
clean
Jan 18, 2025
3bf32b2
update to not always create systemTask, and to use set result descrip…
Jan 18, 2025
845eb97
implement can be restarted
Jan 18, 2025
3b41c8f
wip
Jan 18, 2025
919b634
return error
Jan 20, 2025
f3d4f04
change pointer
Jan 20, 2025
795f8bf
spacing
Jan 20, 2025
36581f7
add rerunActivity endpoint
Jan 20, 2025
feff078
clean up
Jan 21, 2025
94c6e3e
update handler
Jan 21, 2025
e152119
update handler
Jan 21, 2025
5d48771
remove comment
Jan 21, 2025
276ed34
Merge pull request #335 from Receipt-Wrangler/feature/add-recent-acti…
Noah231515 Jan 21, 2025
6cdd855
fix error on fresh redis
Jan 22, 2025
0da1476
update swagger
Jan 24, 2025
d39fc8d
fix error response not being added
Jan 24, 2025
fa79b66
add groupId to system task
Jan 24, 2025
c2d204f
populate group id in system tasks
Jan 24, 2025
09d61e0
populate group id in system tasks pt. 2
Jan 27, 2025
5ad7ade
add receipt id
Jan 27, 2025
2af6075
remove meta combine quick scan
Jan 27, 2025
912794f
remove meta associate to receipt
Jan 27, 2025
008e027
refactor query
Jan 27, 2025
bc16043
associate top level system task back to receipt if successful and fil…
Jan 28, 2025
2bd57e1
update swagger
Jan 28, 2025
2ec5fdb
update poll task to take a payload and refactor poll single group to …
Jan 28, 2025
a75bdcb
make err message more generic
Jan 28, 2025
c90dc9e
update system task receipt and group on create
Jan 28, 2025
13a50d7
make rerun more generic
Jan 28, 2025
132a074
fixes email processor to generate system tasks when failing
Jan 28, 2025
6743914
email upload rerun
Jan 28, 2025
dbac53d
Merge pull request #337 from Receipt-Wrangler/enhancements/rc2-fixes
Noah231515 Jan 29, 2025
85593d1
mk temp dir before test
Jan 29, 2025
6036b18
tidy
Jan 29, 2025
a9fef70
bump package versions
Jan 29, 2025
4fe0957
tidy again
Jan 29, 2025
971a82b
remove repo add
Jan 29, 2025
18a934d
comment...
Jan 29, 2025
f2464c1
removing error
Feb 2, 2025
b27903c
create system task on duplicate
Feb 2, 2025
17ab402
move duplicate to service
Feb 2, 2025
244cf92
Merge pull request #341 from Receipt-Wrangler/bugfix/misc-fixes
Noah231515 Feb 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .air.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
root = "."
testdata_dir = "testdata"
tmp_dir = "tmp"

[build]
args_bin = []
bin = "./tmp/main"
cmd = "go build -o ./tmp/main ."
delay = 1000
exclude_dir = ["assets", "tmp", "vendor", "testdata"]
exclude_file = []
exclude_regex = ["_test.go"]
exclude_unchanged = false
follow_symlink = false
full_bin = ""
include_dir = []
include_ext = ["go", "tpl", "tmpl", "html"]
include_file = []
kill_delay = "0s"
log = "build-errors.log"
poll = false
poll_interval = 0
post_cmd = []
pre_cmd = []
rerun = false
rerun_delay = 500
send_interrupt = false
stop_on_error = false

[color]
app = ""
build = "yellow"
main = "magenta"
runner = "green"
watcher = "cyan"

[log]
main_only = false
silent = false
time = false

[misc]
clean_on_exit = false

[proxy]
app_port = 0
enabled = false
proxy_port = 0

[screen]
clear_on_rebuild = false
keep_scroll = true
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
/temp/*
./api
./coverage.out
./sqlite/*
sqlite/*
.idea/*
./imap-client/__pycache__
api
./dev/mariadb/data
Expand Down
2 changes: 2 additions & 0 deletions dev/switch-to-sqlite.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ export DB_FILENAME=wrangler.db
export DB_ENGINE=sqlite
export ENCRYPTION_KEY=test
export SECRET_KEY=test
export REDIS_HOST=172.17.0.3
export REDIS_PORT=6379
96 changes: 53 additions & 43 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,75 +1,85 @@
module receipt-wrangler/api

go 1.22
go 1.22.7

toolchain go1.23.2

require (
github.com/auth0/go-jwt-middleware/v2 v2.2.1
github.com/gabriel-vasile/mimetype v1.4.6
github.com/alicebob/miniredis/v2 v2.34.0
github.com/auth0/go-jwt-middleware/v2 v2.2.2
github.com/gabriel-vasile/mimetype v1.4.8
github.com/glebarez/sqlite v1.11.0
github.com/go-chi/chi/v5 v5.1.0
github.com/golang-jwt/jwt/v4 v4.5.0
github.com/google/generative-ai-go v0.16.0
github.com/go-chi/chi/v5 v5.2.0
github.com/golang-jwt/jwt/v4 v4.5.1
github.com/google/generative-ai-go v0.19.0
github.com/hibiken/asynq v0.25.1
github.com/jinzhu/copier v0.4.0
github.com/otiai10/gosseract/v2 v2.4.1
github.com/rs/cors v1.11.0
github.com/sashabaranov/go-openai v1.27.1
github.com/rs/cors v1.11.1
github.com/sashabaranov/go-openai v1.36.1
github.com/shopspring/decimal v1.4.0
golang.org/x/crypto v0.31.0
google.golang.org/api v0.187.0
golang.org/x/crypto v0.32.0
golang.org/x/net v0.34.0
google.golang.org/api v0.219.0
gopkg.in/gographics/imagick.v2 v2.7.0
gorm.io/driver/mysql v1.5.7
gorm.io/driver/postgres v1.5.9
gorm.io/gorm v1.25.10
gorm.io/driver/postgres v1.5.11
gorm.io/gorm v1.25.12
)

require (
cloud.google.com/go v0.115.0 // indirect
cloud.google.com/go/ai v0.8.0 // indirect
cloud.google.com/go/auth v0.6.1 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect
cloud.google.com/go/compute/metadata v0.4.0 // indirect
cloud.google.com/go/longrunning v0.5.9 // indirect
cloud.google.com/go v0.118.0 // indirect
cloud.google.com/go/ai v0.10.0 // indirect
cloud.google.com/go/auth v0.14.1 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.7 // indirect
cloud.google.com/go/compute/metadata v0.6.0 // indirect
cloud.google.com/go/longrunning v0.6.4 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/glebarez/go-sqlite v1.22.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sql-driver/mysql v1.8.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.5 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/googleapis/gax-go/v2 v2.14.1 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.6.0 // indirect
github.com/jackc/puddle/v2 v2.2.1 // indirect
github.com/jackc/pgx/v5 v5.7.2 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/redis/go-redis/v9 v9.7.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.53.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 // indirect
go.opentelemetry.io/otel v1.28.0 // indirect
go.opentelemetry.io/otel/metric v1.28.0 // indirect
go.opentelemetry.io/otel/trace v1.28.0 // indirect
golang.org/x/net v0.30.0 // indirect
golang.org/x/oauth2 v0.21.0 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/yuin/gopher-lua v1.1.1 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 // indirect
go.opentelemetry.io/otel v1.34.0 // indirect
go.opentelemetry.io/otel/metric v1.34.0 // indirect
go.opentelemetry.io/otel/trace v1.34.0 // indirect
golang.org/x/exp v0.0.0-20250128182459-e0ece0dbea4c // indirect
golang.org/x/oauth2 v0.25.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/time v0.5.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/grpc v1.65.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
golang.org/x/time v0.9.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250127172529-29210b9bc287 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250127172529-29210b9bc287 // indirect
google.golang.org/grpc v1.70.0 // indirect
google.golang.org/protobuf v1.36.4 // indirect
gopkg.in/go-jose/go-jose.v2 v2.6.3 // indirect
modernc.org/libc v1.54.2 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/sqlite v1.30.1 // indirect
modernc.org/libc v1.61.10 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.8.2 // indirect
modernc.org/sqlite v1.34.5 // indirect
)
Loading