Compare commits

...

1588 Commits

Author SHA1 Message Date
Jon Chambers
08f6ec639c Update to the latest version of the spam filter 2025-12-02 15:59:54 -05:00
Jonathan Klabunde Tomer
6c3cfc88b5 retire /v1/config
It's been supplanted by /v2/config for all clients beyond the 90-day window.
We still have [some traffic](https://signal.grafana.net/goto/bf5tjk346v1moa?orgId=1)
but it's all from expired/third-party clients (note the lack of a recognized
version number in the client-version tag).
2025-12-02 12:52:39 -08:00
Jon Chambers
389d44fd80 Remove legacy delete-via-REST plumbing 2025-12-02 15:45:18 -05:00
Jon Chambers
7604306818 Retire REST-based message deletion 2025-12-02 15:45:18 -05:00
Jon Chambers
92e133b21f Shut down command dependencies in LIFO order 2025-12-02 15:45:01 -05:00
Jon Chambers
4af50986e0 Minor corrections to docs for POST /v1/registration 2025-12-02 15:44:43 -05:00
Jon Chambers
c72458b47a Perform basic input validation on call quality survey responses 2025-12-01 09:56:09 -05:00
Jon Chambers
aa2f9e5a65 Accept start/end timestamps as milliseconds since the epoch 2025-11-26 09:00:54 -05:00
Jonathan Klabunde Tomer
f13837d2f2 improve OpenTelemetry logging 2025-11-25 12:23:00 -08:00
Jonathan Klabunde Tomer
3ff2af47cb Revert "support JSON logging and prefer it in sample config"
This reverts commit 4d5cc4dc22.
2025-11-25 12:23:00 -08:00
Jon Chambers
c719da3527 Update to the latest version of the spam filter 2025-11-25 12:22:38 -05:00
Jon Chambers
0ea52b785e Update to the latest version of the spam filter 2025-11-25 11:27:05 -05:00
Jon Chambers
1ce1c298d3 Add hooks so spam filters can register Dropwizard commands 2025-11-25 11:07:57 -05:00
Jonathan Klabunde Tomer
4d5cc4dc22 support JSON logging and prefer it in sample config
We won't actually remove OTLP logging support until testing side-by-side and
seeing that things look good.
2025-11-24 16:05:28 -08:00
Jon Chambers
640274108e Make CommandDependencies public 2025-11-24 10:47:00 -05:00
Jon Chambers
dd17ddc98c Add audio/video variants to several call quality survey metrics 2025-11-24 08:48:52 -05:00
Jonathan Klabunde Tomer
4c4a954c1c update shutdown gauge when delayed shutdown starts
Otherwise we will report that we are not shutting down while k8s correctly
notes that we are unhealthy and it will look like something is wrong.
2025-11-19 16:43:48 -08:00
Katherine
65ce9af366 Retire ssre2 and deleteSync capabilities 2025-11-18 12:42:13 -05:00
Chris Eager
4af0de2ab2 Minor cleanup 2025-11-18 11:39:54 -06:00
Chris Eager
ccf72a45db Migrate Stripe usage to V1Services 2025-11-18 11:39:54 -06:00
Chris Eager
f3744fbcb1 Address deprecated methods 2025-11-18 11:39:54 -06:00
Chris Eager
ec08731e6d Use appropriate expansion for latest invoice -> charge 2025-11-18 11:39:54 -06:00
Chris Eager
852591df40 Update stripe-java to 30.2.0 2025-11-18 11:39:54 -06:00
Chris Eager
9ff9b3a7b3 Fix erroneous parameter doc in MessageController#sendMessage 2025-11-10 12:27:14 -06:00
Chris Eager
dd4e058cd7 Update response schema annotation for MessageController#sendMessage 2025-11-10 12:08:59 -06:00
Ravi Khadiwala
85226bdd87 Retrieve subscription price from purchase instead of plan 2025-11-07 11:30:05 -06:00
Ravi Khadiwala
faa74469ea Revert "Update to Dropwizard 5"
This reverts commit 4dbd564442.
2025-11-06 17:20:41 -06:00
Jonathan Klabunde Tomer
bb94975d74 remove datadog/statsd support 2025-11-06 09:24:55 -08:00
Jon Chambers
298b0d8d28 Add new variants of call quality survey values and clarify optionality 2025-11-06 12:23:52 -05:00
Jonathan Klabunde Tomer
9da643dd69 update to the latest revision of the spam filter 2025-11-05 12:07:36 -08:00
Jonathan Klabunde Tomer
6dc4bfe5fa Update legacy metric names
We're taking a massive history cliff when we move off Datadog anyway; let's
take the opportunity to remove all the old-school
`org.whispersystems.some.long.path.SomeClass.metric` names in favor of the newer
`chat.SomeClass.metric` style, and update any metrics that were moved from one
class to another and kept the old name for continuity's sake.
2025-11-05 12:07:36 -08:00
Jon Chambers
23a3e32eb8 Tag client request metrics with specific client versions 2025-11-04 16:29:23 -05:00
Ravi Khadiwala
bf6939ec00 Add platform tag to backup auth metric 2025-11-04 16:29:02 -05:00
Katherine Yen
3b310463c3 Update to the latest version of the spam filter 2025-11-04 16:27:16 -05:00
Katherine
f7eb6fab33 Prefix AssertTrue validation methods with "is" 2025-11-04 16:25:12 -05:00
Andrew
ce945ff245 Add flag to negotiate server_no_context_takeover 2025-11-04 12:21:36 -06:00
ravi-signal
4dbd564442 Update to Dropwizard 5
Co-authored-by: Chris Eager <chris@signal.org>
2025-11-04 12:18:56 -06:00
Ravi Khadiwala
24f8f48a26 Check sandbox when storekit transactionId is not found 2025-10-30 09:34:17 -05:00
Jon Chambers
a2ce37fd53 Limit concurrency when unlinking devices 2025-10-30 10:32:01 -04:00
Jon Chambers
c4d55e099e Remove enrollment fraction plumbing from UnlinkDevicesWithIdlePrimaryCommand 2025-10-30 10:32:01 -04:00
Katherine
0f950917d8 Fix key transparency monitor request position validation 2025-10-29 16:34:33 -04:00
Katherine
3116913378 Support searching for identifiers in the key transparency log by version 2025-10-29 16:22:55 -04:00
Jonathan Klabunde Tomer
d7a9e3c9f3 Don't send TLS session resumption tickets 2025-10-29 16:22:38 -04:00
Jon Chambers
14010a4f83 Move to the Bitnami Legacy Redis Cluster image 2025-10-29 16:21:13 -04:00
Jon Chambers
ad0bcd5436 Add a command for unlinking devices linked to an account whose primary device is idle 2025-10-24 15:23:57 -04:00
Jon Chambers
88d458cf79 Convert RegistrationLock to a record 2025-10-24 15:23:12 -04:00
Katherine Yen
342c8a1b28 Check for an authorization header in PUT /v1/devices/link 2025-10-24 15:23:01 -04:00
Jon Chambers
ad2500d4fd Fix an incorrect documentation tag for the call quality survey controller 2025-10-24 15:22:46 -04:00
Jon Chambers
c2ebabad58 Work around an AWS SDK stack overflow issue 2025-10-21 17:01:20 -04:00
Jon Chambers
1105753aab Update to the latest version of the spam filter 2025-10-16 14:36:31 -04:00
Jon Chambers
9378b9a6e6 Add controllers/service implementations for receiving call quality survey responses 2025-10-16 14:22:12 -04:00
Jon Chambers
c68e3103c4 Introduce CallQualitySurveyManager 2025-10-16 14:22:12 -04:00
Jon Chambers
c9760f4c38 Introduce a system for mapping IP addresses to ASNs 2025-10-16 14:22:12 -04:00
Chris Eager
73765fc4ec Update variable names in recently refactored method 2025-10-16 12:12:46 -05:00
Jon Chambers
9e1b716548 Fix a need-to-wait-for-async-operation issue in testCacheRemoveQueueScript 2025-10-10 17:03:58 -04:00
ravi-signal
a2f2fc93b0 Add direct grpc server 2025-10-06 15:22:36 -05:00
ravi-signal
9751569dc7 Marshal HttpResponseException in GooglePlayBillingManager 2025-10-06 12:18:51 -05:00
ravi-signal
d6c15ef1d5 Allow rotating a single backup-id at a time 2025-10-06 12:18:31 -05:00
Chris Eager
e0eaa76ebf Minor improvements 2025-10-06 12:15:11 -05:00
Chris Eager
0d4d9c0af5 Set request timeouts on Fixer and CoinGecko 2025-10-06 12:15:11 -05:00
Chris Eager
4ffd768aac Use shared cache for Fixer data 2025-10-06 12:15:11 -05:00
Chris Eager
b8a720c5b4 Factor Coin Gecko caching to methods 2025-10-06 12:15:11 -05:00
Chris Eager
ab1ec86cd2 Decompose CurrencyConversion update methods 2025-10-06 12:15:11 -05:00
Chris Eager
e9d6b91416 Add Managed#stop to CurrencyConversionManager 2025-10-06 12:15:11 -05:00
Ameya Lokare
63f892add1 Update to the latest version of the spam filter 2025-10-01 15:19:03 -07:00
Chris Eager
6e42b2898c Make BraintreeConfiguration.publicKey a SecretString 2025-10-01 13:06:56 -05:00
ravi-signal
9384813752 Forgive some clock skew when requesting ZK credentials 2025-10-01 13:03:27 -05:00
Chris Eager
70ac4ad139 Update response documentation for PUT /v1/challenge 2025-10-01 12:59:48 -05:00
Chris Eager
4ab58e950b Add authenticated tag to MetricsRequestEventListener 2025-09-29 15:37:24 -07:00
Jonathan Klabunde Tomer
7f301cbf95 filter datadog metrics with dynamic allow list 2025-09-29 15:37:11 -07:00
Ameya Lokare
5a7660d3ae Update to the latest version of the spam filter 2025-09-29 11:13:50 -07:00
Chris Eager
850172f6a3 Register system resource metrics in MetricsUtil#configureRegistries 2025-09-29 11:11:17 -07:00
Ravi Khadiwala
4e73162055 Fix default maxQuotaStaleness 2025-09-29 13:07:21 -05:00
Jonathan Klabunde Tomer
4127cf90ab also set the OTLP logging resource from config 2025-09-22 14:10:11 -07:00
Jonathan Klabunde Tomer
08cc7bc462 use standard OTLP endpoint environment variable for logging 2025-09-22 12:50:55 -07:00
ravi-signal
2cf2391f38 Avoid NPE validating invalid RegistrationRequest 2025-09-22 11:23:59 -07:00
Jonathan Klabunde Tomer
007dde8d45 add OTLP logging appender factory 2025-09-22 11:09:40 -07:00
Katherine Yen
f80e30f9f2 Update to the latest version of the spam filter 2025-09-18 14:57:52 -04:00
Katherine
ff5201a58e Q4 2025 dependency updates 2025-09-18 14:56:27 -04:00
Chris Eager
8954708d77 Update OptionalAccess check with PNI service IDs
This was mis-aligned with other parts of the service and client implementations
2025-09-18 13:54:11 -05:00
Katherine Yen
3cf194e476 Catch IllegalStateException when closing RecordingStream 2025-09-18 13:53:31 -05:00
Jordan Rose
bad2602491 test.yml: Use libsignal's test sealed sender certificate 2025-09-18 13:52:37 -05:00
Ravi Khadiwala
a647c1bfdf Update backup media metrics on all TTL refreshes 2025-09-18 13:52:13 -05:00
Ravi Khadiwala
1bba30a81e Make per-message deflate extension for websockets configurable 2025-09-18 13:51:36 -05:00
Chris Eager
827d8388d0 Fix comment typos in test.yml 2025-09-17 10:58:34 -05:00
Jordan Rose
70efb19a86 test.yml: Move sealed sender certificate from secrets bundle 2025-09-17 10:45:50 -05:00
Ravi Khadiwala
35ffb208e3 Make max total backup media configurable 2025-09-15 13:03:42 -05:00
Chris Eager
e50dcd185d Update stripe-java to 28.4.0 2025-09-15 13:01:17 -05:00
Ravi Khadiwala
1cd446ee31 Add paid tag to BackupMetricsCommand counter 2025-09-12 16:41:46 -05:00
Chris Eager
1847c6d4ca Update to the latest version of the spam filter 2025-09-12 16:39:51 -05:00
ravi-signal
1770558d5e Add an endpoint to check if your backup-id can be rotated
Co-authored-by: Katherine <katherine@signal.org>
2025-09-12 16:39:01 -05:00
Chris Eager
e0d39212ec Update to the latest version of the spam filter 2025-09-11 16:24:59 -05:00
Ravi Khadiwala
31ebe9071a Don't count backup metrics on TTL refresh 2025-09-11 16:20:51 -05:00
Chris Eager
8f65486fab Update BaseRateLimiters#defaultScript 2025-09-11 13:40:35 -05:00
Ravi Khadiwala
7e3540bda0 Workaround for sdk issue iterating large DynamoDB pages 2025-09-10 11:05:29 -05:00
ravi-signal
c544628dfe Add a command to clear IAP issued receipt count
Co-authored-by: Katherine <katherine@signal.org>
2025-09-10 11:00:02 -05:00
Ravi Khadiwala
61b162d0a1 clean up warnings in BackupManagerTest 2025-09-10 10:59:47 -05:00
Ravi Khadiwala
8c2d738924 Make backup batch operation concurrency configurable 2025-09-10 10:59:47 -05:00
Ravi Khadiwala
efde8a31f9 Missing Money.nanos should be treated as 0 2025-09-08 15:21:43 -05:00
Katherine
b2dd315177 Set TCP timeout on Redis clients 2025-09-08 12:37:03 -04:00
Jordan Rose
dc3920a99c Avoid setting the isStory field in an Envelope when possible 2025-09-05 15:33:38 -05:00
Ameya Lokare
b8e8fd3313 Implement MRM insert in FoundationDB message store 2025-09-04 11:59:28 -07:00
ravi-signal
a5423b6e21 Simplify SubscriptionExceptions 2025-09-04 13:50:51 -05:00
Jordan Rose
89b37015c6 Reserve a second sealed sender ServerCertificate ID 2025-09-04 13:49:15 -05:00
Ravi Khadiwala
2af3088571 Preserve backup vouchers accross re-registration 2025-09-04 13:48:45 -05:00
ravi-signal
774cc52b61 Make SubscriptionController synchronous 2025-09-02 15:11:05 -05:00
Ameya Lokare
f52a262741 Update to the latest version of the spam filter 2025-08-28 13:43:53 -07:00
Jon Chambers
cd957e0347 Standardize circuit breaker/retry names 2025-08-28 13:38:25 -04:00
Jon Chambers
8060d74899 Automatically register metrics/instrument breaker/retry instances on creation 2025-08-28 13:38:25 -04:00
Jon Chambers
0e1e248564 Remove resilience4j-micrometer 2025-08-28 13:38:25 -04:00
Jon Chambers
d33761f107 Revert "reduce cost of resilience4j metrics"
This reverts commit 8e429e267f.
2025-08-28 13:38:25 -04:00
Jon Chambers
89c7521be0 Don't request more messages if we already have an active message source 2025-08-28 13:26:33 -04:00
Jonathan Klabunde Tomer
8e429e267f reduce cost of resilience4j metrics 2025-08-27 18:02:48 -07:00
Jon Chambers
8fe87b77e4 Wait for message acknowledgement before fetching new messags from Redis/DynamoDB 2025-08-27 15:20:05 -04:00
Jon Chambers
194e43926a Reapply "Move error/entry handling to subscribe handlers"
This reverts commit 4d521cea42.
2025-08-27 15:20:05 -04:00
Jon Chambers
4c0281f540 Reapply "Add a (failing!) test for dropped errors during WebSocketConnectionTest"
This reverts commit 0f2a4d02e0.
2025-08-27 15:20:05 -04:00
Jon Chambers
75391785f8 Reapply "Use MessageStream in WebSocketConnection"
This reverts commit a94ce72894.
2025-08-27 15:20:05 -04:00
Jon Chambers
ebdc5a30f8 Restore "may have messages" methods 2025-08-27 15:20:05 -04:00
Jon Chambers
f57093a94a Prefix breaker/retry names where appropriate 2025-08-27 14:45:29 -04:00
Jon Chambers
53f9c7b31f Remove unnecessary "-breaker" and "-retry" suffixes from names 2025-08-27 14:45:29 -04:00
Jon Chambers
bc20aee7c9 Rename CircuitBreakerUtil to ResilienceUtil 2025-08-27 14:45:29 -04:00
Jon Chambers
807e03ca2b Remove legacy circuit breaker/retry metrics 2025-08-27 14:45:29 -04:00
Jon Chambers
a747afb487 Publish metrics from circuit breaker/retry registries 2025-08-27 14:45:29 -04:00
Jon Chambers
eb42d22155 Update to the latest version of the spam filter 2025-08-27 12:05:30 -04:00
Jon Chambers
8825396fc1 Restore Redis retries for select operations 2025-08-27 11:52:16 -04:00
Jon Chambers
f616612104 Use central registries for Retry and CircuitBreaker instances 2025-08-27 11:33:42 -04:00
Jon Chambers
a8c6fa93e0 Remove default/integral retries from Redis clients 2025-08-27 11:20:11 -04:00
Jon Chambers
32cf12e9af Remove an unused method from RetryConfiguration 2025-08-27 11:20:11 -04:00
Jon Chambers
1cc933b6bc Pause if we hit an exception while processing scheduled notifications 2025-08-27 11:05:17 -04:00
Ravi Khadiwala
dbbd913445 Allow downgrade on SQPR capability 2025-08-22 11:50:55 -05:00
Jon Chambers
50e7301dd8 Remove cluster-specific configuration from Redis singleton clients 2025-08-22 12:46:32 -04:00
Jonathan Klabunde Tomer
552edbc1eb update to the latest revision of the spam filter 2025-08-21 15:25:40 -07:00
Jon Chambers
7f5ea6608c Add plumbing to roll out binary service IDs/UUIDs on envelopes to internal users 2025-08-21 14:53:41 -07:00
Jonathan Klabunde Tomer
78a7112675 Use native exponential histograms 2025-08-21 14:53:21 -07:00
Ravi Khadiwala
be8b44d645 Add noise tunnel connection metrics 2025-08-21 10:32:11 -05:00
Ravi Khadiwala
7ca3604601 Augment gRPC metrics with UA 2025-08-21 10:32:11 -05:00
Ravi Khadiwala
c5af8f3a9e Adjust default backup level configuration 2025-08-19 12:43:51 -05:00
Ravi Khadiwala
d67847e5b1 Make s3 endpoints configurable 2025-08-19 12:29:42 -05:00
Jon Chambers
3a5704a5cc Fix a flaky MessageCacheInsertScript test by calling .join() on async operations 2025-08-19 13:14:20 -04:00
Katherine Yen
50a54136e8 Update to the latest version of the spam filter 2025-08-18 14:59:07 -04:00
ameya-signal
a1d9c4c062 Check presence before updating last message versionstamp 2025-08-18 10:16:00 -07:00
Katherine
4acb3b5ac7 Use the target device's registration ID in the GET /v1/keys/identifier/deviceId ratelimit 2025-08-18 10:15:45 -07:00
ravi-signal
b76eaa1098 Add a bounded virtual executor service 2025-08-15 15:49:50 -05:00
Ravi Khadiwala
c883cd8148 Encourage PQ prekey upload to new store 2025-08-15 15:48:57 -05:00
Ravi Khadiwala
1e77d0471e Remove paged prekey experiment configuration 2025-08-15 15:48:57 -05:00
Jon Chambers
a94ce72894 Revert "Use MessageStream in WebSocketConnection"
This reverts commit 470e17963a.
2025-08-13 16:13:50 -04:00
Jon Chambers
0f2a4d02e0 Revert "Add a (failing!) test for dropped errors during WebSocketConnectionTest"
This reverts commit fc8d180f7c.
2025-08-13 16:13:50 -04:00
Jon Chambers
4d521cea42 Revert "Move error/entry handling to subscribe handlers"
This reverts commit 7d10209198.
2025-08-13 16:13:50 -04:00
Jon Chambers
7d10209198 Move error/entry handling to subscribe handlers 2025-08-13 14:38:53 -04:00
Jon Chambers
fc8d180f7c Add a (failing!) test for dropped errors during WebSocketConnectionTest 2025-08-13 14:38:53 -04:00
Jon Chambers
470e17963a Use MessageStream in WebSocketConnection 2025-08-13 10:22:55 -04:00
Chris Eager
4c5dc118aa Use nanoTime for queue drain metric 2025-08-11 18:26:40 -04:00
Chris Eager
ad84cd848a Fix potential NullPointerException in StripeManager#createPaymentIntent 2025-08-11 18:26:21 -04:00
Katherine
127bd56f73 Don't require ALWAYS_CAPABLE capabilities when linking a device 2025-08-11 18:25:55 -04:00
Jon Chambers
ad3721acf1 Count "send sync message" attempts with incorrect registration IDs 2025-08-11 18:24:38 -04:00
Jon Chambers
085127326b Improve thread synchronization in RedisDynamoDbMessagePublisherTest 2025-08-11 18:24:10 -04:00
Jon Chambers
2e32ab3282 Extend the Redis command timeout in FaultTolerantRedisClusterClientTest 2025-08-11 18:23:51 -04:00
ameya-signal
8bb125597b Add initial version of FoundationDB message store 2025-08-11 18:23:27 -04:00
Jonathan Klabunde Tomer
8ca7f85bf0 fix NPE in MetricsUtil#configureHistogramFilters 2025-08-11 14:11:17 -07:00
Jonathan Klabunde Tomer
9d4d36939c Update to the latest revision of the spam filter 2025-08-11 12:30:40 -07:00
Jonathan Klabunde Tomer
dabe1c157b remove unnecessary MessageMetrics default constructor 2025-08-11 12:28:28 -07:00
Jonathan Klabunde Tomer
36d0c4422e Configure Micrometer distribution buckets for OpenTelemetry
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2025-08-11 11:45:33 -07:00
Jon Chambers
1429efd573 Retire duplicative "message delivery time" metrics 2025-08-08 13:52:20 -04:00
Katherine
93c03c5676 Hardcode ssre2 and deleteSync capabilities to true from GET /v1/profile/{identifier} 2025-08-08 09:28:13 -04:00
Katherine Yen
bd1bd007c0 Count successful timezone parsing to get a ratio of success to failures 2025-08-08 09:27:46 -04:00
Katherine Yen
f485406c4d Stop using device creation timestamp as a fallback in getting a recommended notification time 2025-08-08 09:27:46 -04:00
Jon Chambers
d96c360016 Introduce MessageStream and RedisDynamoDbMessagePublisher 2025-08-08 09:27:05 -04:00
Jon Chambers
6d71f43b6c Update to the latest version of the spam filter 2025-08-08 09:26:21 -04:00
Katherine
d6824fcaf6 Remove EncryptDeviceCreationTimestampCommand 2025-08-04 12:51:07 -04:00
Katherine
61f388f5f4 Rename registrationId field on TransferArchiveUploadedRequest 2025-08-04 12:50:56 -04:00
Jon Chambers
6955e1ee20 Retire RedisClusterHealthCheck 2025-08-04 09:59:28 -04:00
Jon Chambers
7bb4a4bc90 Remove an unused hasMessages method 2025-08-04 09:59:17 -04:00
Jon Chambers
6fa01a5d48 Remove unused Lua scripts 2025-08-04 09:59:17 -04:00
Jon Chambers
5a65a46fc1 Remove an unused constant 2025-08-04 09:59:17 -04:00
Jon Chambers
c255355e5c Make internal Redis connection management methods private 2025-08-04 09:59:17 -04:00
Jon Chambers
8d3781db90 Remove the "send message" timeout 2025-08-04 09:58:56 -04:00
ravi-signal
f6a87b2ec0 Always elide devices missing required PreKeys 2025-08-01 12:01:35 -05:00
ravi-signal
e8a1854c5e Move /v1/svrb/auth to /v1/archives/auth/svrb 2025-08-01 12:00:44 -05:00
Katherine
f8d27d8fab "encrypt timestamps" crawler optimizations 2025-08-01 12:31:09 -04:00
Jon Chambers
94dd2712b1 Extent timeout in WebSocketConnectionTest 2025-07-31 17:09:02 -04:00
Jon Chambers
a4804f6501 Don't retry "connection closed" errors 2025-07-31 16:37:19 -04:00
Jon Chambers
5cb3a053fb Add a timer for sendMessage calls 2025-07-31 16:37:19 -04:00
Jon Chambers
4923b6da68 Measure only errors that terminate a message stream 2025-07-31 16:37:19 -04:00
Jon Chambers
b7e64e09a3 Add an already-passing, disabled-by-default test for client WebSocket timeouts 2025-07-31 16:37:19 -04:00
Jon Chambers
76e65a47a2 Represent timeouts as Durations rather than bare millis 2025-07-31 16:37:19 -04:00
Jon Chambers
ef644ce319 Drop WhisperServerServiceTest#testOtlpConfig 2025-07-31 16:27:29 -04:00
Jon Chambers
08c4179a7a Disable OpenTelemetry in tests 2025-07-31 16:27:29 -04:00
Jon Chambers
5c3be9c3d6 Use error-specific retry mechanisms in WebSocketConnection and associated classes 2025-07-31 10:53:11 -04:00
Jonathan Klabunde Tomer
8fc0b49994 tweak OpenTelemetry configuration 2025-07-30 14:42:20 -07:00
Katherine
db4c71368c Use registration ID or creation timestamp in the transfer archive flow 2025-07-30 15:32:49 -04:00
Jon Chambers
30774bbc40 Remove special-case handling for an extremely narrow message delivery edge case 2025-07-29 16:52:19 -04:00
Ravi Khadiwala
3b48c82c72 Update to the latest version of the spam filter 2025-07-29 15:51:22 -05:00
Jon Chambers
038c68c594 Rename WebSocketConnectionEventManager/Listener to MessageAvailabilityManager/Listener 2025-07-29 11:29:17 -04:00
Jon Chambers
cf222e1105 Scope disconnection request listeners to a single connection 2025-07-29 11:17:48 -04:00
Jon Chambers
541c87e262 Always specify a list of device IDs when disconnecting all devices for an account 2025-07-29 11:17:48 -04:00
Katherine
5f5c345f94 Use one thread per account in the "encrypt timestamps" crawler 2025-07-29 09:51:36 -04:00
Jonathan Klabunde Tomer
8aa408a3c1 enable OTLP export 2025-07-28 14:42:37 -07:00
Jonathan Klabunde Tomer
4f0337021c Use Micrometer rather than Dropwizard for a few metrics 2025-07-28 14:32:42 -07:00
Jonathan Klabunde Tomer
00062fdd5c Revert "Use Micrometer rather than Dropwizard for a few metrics"
This reverts commit 2bc91c1f21.
2025-07-25 10:44:42 -07:00
Jonathan Klabunde Tomer
2bc91c1f21 Use Micrometer rather than Dropwizard for a few metrics 2025-07-25 10:17:50 -07:00
Ravi Khadiwala
c92a29db1e Update to the latest version of the spam filter 2025-07-25 12:08:53 -05:00
Ravi Khadiwala
37d67f110a Handle unexpectedly missing last-resort prekeys 2025-07-25 11:54:50 -05:00
Katherine
bf779f30ab Fix number of accounts processed counter 2025-07-24 12:26:49 -04:00
Jon Chambers
4d81124dfa Remove redundant disconnection requests 2025-07-24 10:33:10 -04:00
Jonathan Klabunde Tomer
ccf8840fa3 fix accidental reversal of metrics 2025-07-24 10:32:34 -04:00
Katherine
267aafe861 Encrypt device creation timestamp crawler 2025-07-24 10:29:52 -04:00
Chris Eager
1bdcfb1d83 Don’t unnecessarily reinitialize test DynamoDB client 2025-07-23 13:32:58 -05:00
Chris Eager
0c69ef9381 Update to the latest version of the spam filter 2025-07-23 13:18:39 -05:00
Chris Eager
c99b1cada1 Add overrides configuration to LocalDynamoDbFactory 2025-07-23 13:13:23 -05:00
Chris Eager
83d19ac8ed Update localstack test container to 4.x 2025-07-23 13:13:23 -05:00
Chris Eager
5f77d7f582 Migrate from embedded DynamoDBLocal to Testcontainers 2025-07-23 13:13:23 -05:00
Katherine
96f6e75702 Introduce encrypted device creation timestamps 2025-07-23 10:36:11 -04:00
Katherine
74c7e49cea Replace device creation timestamps with registration IDs in experiment logic 2025-07-23 10:24:28 -04:00
Jon Chambers
876bf15a11 Extend timeouts for Redis client tests 2025-07-23 10:24:07 -04:00
Chris Eager
de60752219 Convert Profiles gRPC to SimpleGrpc 2025-07-23 10:23:48 -04:00
Katherine
5d80ac73da Update libsignal to 0.76.5 2025-07-23 10:22:48 -04:00
Ravi Khadiwala
0ff32d5cae Update to the latest version of the spam filter 2025-07-21 12:01:45 -05:00
Chris Eager
4618b47141 Convert classes using @JsonUnwrapped to records 2025-07-21 11:52:25 -05:00
Ravi Khadiwala
94361b2d5d Remove allow list from pin event monitor 2025-07-21 11:50:30 -05:00
Ravi Khadiwala
d4429ebce1 Update to Java 24 and add .java-version 2025-07-21 11:50:30 -05:00
Jonathan Klabunde Tomer
b5711ead25 /v2/config: don't return configs namespaced to different platforms
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2025-07-18 14:59:41 -07:00
ravi-signal
522ddd4e61 Add documentation for 413 response from /v1/archives/upload/form 2025-07-18 14:59:26 -07:00
Jon Chambers
73365369df Fix a silly typo when generating Redis URIs for tests 2025-07-17 13:16:14 -04:00
Jon Chambers
631b9a5290 Measure the rate of new WebSocket connections by authentication type 2025-07-17 11:35:15 -04:00
Jon Chambers
4ccd39fd55 Validate intra-account messages before applying number changes 2025-07-17 11:34:50 -04:00
Jon Chambers
50bc6b2c62 Update to the latest version of the spam filter 2025-07-17 11:33:46 -04:00
Jon Chambers
609b86acb9 Inline the cluster compose file 2025-07-17 11:17:40 -04:00
Jon Chambers
1ea84483da Copy Redis cluster compose file to a temporary file 2025-07-17 11:17:40 -04:00
Jon Chambers
ed5086823c Extend timeout for MessagePersisterTest 2025-07-17 10:00:48 -04:00
Jon Chambers
73748a6341 Create new Redis client resources for each test 2025-07-17 10:00:48 -04:00
Jon Chambers
1f60300555 Remove embedded-redis as a dependency 2025-07-17 10:00:48 -04:00
Jon Chambers
85a13a9dc0 Extend lifecycle of Redis cluster clients 2025-07-17 10:00:48 -04:00
Jon Chambers
803e73bd1e Use a Redis testcontainer instead of embedded-redis in RedisClusterExtension 2025-07-17 10:00:48 -04:00
Jon Chambers
a3356d0188 Use a Redis testcontainer instead of embedded-redis in RedisServerExtension 2025-07-17 10:00:48 -04:00
Jonathan Klabunde Tomer
5c21aa2ad4 implement /v2/config API (#2764) 2025-07-16 15:29:31 -07:00
Jon Chambers
6116830da9 Remove spurious null checks 2025-07-16 12:34:20 -04:00
Jon Chambers
650bc2598b Retire AccountsManager#updatePniKeys and associated plumbing 2025-07-16 12:34:20 -04:00
Jon Chambers
a36fba061a Remove "no action on change to same number" optimization for "change number" operations 2025-07-16 12:34:20 -04:00
Jon Chambers
e62b3d390f Accquire pessimistic account locks by sets of identifiers instead of lists 2025-07-16 12:34:20 -04:00
Jon Chambers
1a8ebf80b5 Add an integration test for changing to the same phone number 2025-07-16 12:34:20 -04:00
Jon Chambers
2564f706d8 Retire the "distribute PNI key material" endpoint 2025-07-16 12:34:20 -04:00
Ravi Khadiwala
c97c00bf5f Fix the play mapping to stripe statuses
CANCELLED in play terminiology is the same as an active subscription
with cancelAtPeriodEnd set. So it should map to the ACTIVE stripe
status. EXPIRED is the equivalent of a stripe CANCELLED subscription.
2025-07-16 11:19:15 -05:00
Ravi Khadiwala
702b125a48 Accept all entitled play tokens 2025-07-16 11:19:15 -05:00
Chris Eager
ca9f29f984 Only delete profile avatars during explicit delete actions
This preserves the avatar during re-registration, when PIN recovery might occur.
2025-07-15 16:19:52 -05:00
ravi-signal
58b9fa100d Remove from svrb on account deletion 2025-07-15 16:17:14 -05:00
ravi-signal
656b08f3b6 Avoid potential concurrent modification in KeysController 2025-07-15 16:15:08 -05:00
Ravi Khadiwala
3f62677176 Allow optional size parameter when requesting message backup upload forms 2025-07-15 16:14:33 -05:00
Ravi Khadiwala
ae2d98750c Add SecureValueRecoveryBController 2025-07-11 13:39:18 -05:00
Ravi Khadiwala
7d41c1219b Add /v2/svr as an alternative name for /v2/backup 2025-07-11 13:39:18 -05:00
Ravi Khadiwala
65e1f1b3a9 Arm the RemoveOrphanedPreKeyPagesCommand 2025-07-11 13:26:15 -05:00
Ameya Lokare
437b823c84 Update to the latest version of the spam filter 2025-07-09 13:27:04 -07:00
ravi-signal
c9f21d5970 Always read from new and old PQ prekey stores, add experiment to start writing to new prekey store 2025-07-09 09:17:17 -05:00
Ravi Khadiwala
80c11e7eda Handle 429s from play API and add subscription docs 2025-07-09 09:15:29 -05:00
Jon Chambers
0745cabc87 Explicitly use synchronous flush mode when clearing Redis databases between tests 2025-07-09 09:15:15 -05:00
Jon Chambers
3e80669f4e Reuse/extend lifetime of Redis singleton resources 2025-07-09 09:15:15 -05:00
Jon Chambers
b81cd9ec61 Reuse Redis clusters for the duration of a test run 2025-07-09 09:15:15 -05:00
Jon Chambers
da6ed94443 Reuse client resources for lifetime of Redis cluster 2025-07-09 09:15:15 -05:00
Ameya Lokare
96d41b3716 Update to the latest version of the spam filter 2025-07-07 09:16:48 -07:00
Ravi Khadiwala
7dddc4d759 fix an incorrect backup metric 2025-07-07 18:14:53 +02:00
Katherine
a87690d817 Include Redis cluster and shard address in circuit breaker log 2025-07-07 12:12:44 -04:00
Ameya Lokare
18ef3da261 Update dependencies 2025-06-30 14:17:03 -07:00
Ameya Lokare
f4698dd5b2 Update to the latest version of the spam filter 2025-06-27 12:07:45 -07:00
Adel Lahlou
d4322a2ed4 Remove latency based 1:1 call routing 2025-06-27 12:06:43 -07:00
Jon Chambers
7260a9d5b4 Make FoundationDB versions available at runtime 2025-06-27 11:21:50 -04:00
Jon Chambers
12b4ceb4aa Configure FoundationDB service container's database via Docker, removing fdbcli dependency 2025-06-27 11:08:58 -04:00
Jon Chambers
fa1cd5c263 Install the Maven-fetched FoundationDB client library on GitHub Actions runner 2025-06-27 11:06:04 -04:00
Jon Chambers
f8da13912d Fetch the FoundationDB client library as a pre-package step rather than including it in version control 2025-06-27 11:04:53 -04:00
Jon Chambers
a3b3bf86ba Add a note about the FoundationDB client library requirement to the README 2025-06-27 11:04:52 -04:00
Jon Chambers
a99f7bb87d Add test dependencies for FoundationDB 2025-06-27 11:04:52 -04:00
Jon Chambers
d6f14d02dd Add a FoundationDB service container for tests 2025-06-27 11:04:46 -04:00
Jon Chambers
d18671eaf9 Add FoundationDB runtime dependencies 2025-06-26 12:13:09 -04:00
Jon Chambers
87c30d00e8 Store compressed envelopes at rest 2025-06-25 15:20:19 -04:00
Jon Chambers
c8f45685b8 Expand envelopes on load from storage 2025-06-25 14:31:19 -04:00
Jon Chambers
bb90d80d22 Add a utility for compressing/expanding envelopes 2025-06-25 14:31:19 -04:00
Jon Chambers
dcc541f86e Add binary representation fields for service IDs/UUIDs 2025-06-25 14:31:19 -04:00
Ravi Khadiwala
aaa36fd8f5 Add a crawler for orphaned prekey pages 2025-06-24 13:46:48 -05:00
Ravi Khadiwala
2bb14892af Add paged prekey store 2025-06-24 13:46:48 -05:00
Ameya Lokare
6d8701665e Update to the latest version of the spam filter 2025-06-24 11:46:11 -07:00
Katherine
c2b8fdac0d Only log for an unexpected error from the key transparency service 2025-06-24 14:45:53 -04:00
Katherine
059caa4c57 Implement key transparency endpoints using simple-grpc 2025-06-24 14:01:35 -04:00
Jon Chambers
51773f5709 Update to the latest version of the spam filter 2025-06-23 10:20:24 -04:00
Jon Chambers
483404a67f Retire authenticated device getters 2025-06-23 10:10:30 -04:00
Jon Chambers
68b84dd56b Remove the PQ key check from IdlePrimaryDeviceAuthenticatedWebSocketUpgradeFilter
All devices now have PQ keys
2025-06-23 08:58:15 -05:00
Jon Chambers
7709e1313c Update to the latest version of the spam filter 2025-06-23 09:50:45 -04:00
Jon Chambers
c952baa672 Don't cache authenticated accounts in memory 2025-06-23 09:40:05 -04:00
Ravi Khadiwala
9dfe51eac4 Forbid linked devices from setting backup-ids 2025-06-18 11:07:52 -05:00
andrew-signal
5de848bf38 Instrument request/response sizes 2025-06-17 11:16:57 -04:00
Ravi Khadiwala
295cedc075 remove experiment configuration for low urgency pushes 2025-06-17 09:43:35 -05:00
Jon Chambers
4f1cab407f Simplify WebSocket authentication failure handling 2025-06-17 10:41:29 -04:00
Ravi Khadiwala
626a7fdad7 Add docs to /v1/donations/redeem-receipt 2025-06-12 17:07:19 -05:00
Jon Chambers
9a1da23bdb Add an isEphemeral dimension to message delivery latency metrics 2025-06-10 17:05:46 -05:00
ravi-signal
4ffd164461 Wire up the direct noise tunnel 2025-06-10 16:56:31 -05:00
Jon Chambers
904cc63a72 Clarify that OutgoingMessageEntity#toEnvelope is a test-only method 2025-06-10 16:55:13 -05:00
Ravi Khadiwala
177c36b0d6 Fix backup metric names and use remote aggregation 2025-06-10 16:54:45 -05:00
Jon Chambers
5fc6bdd478 Add a device capability for sparse post-quantum ratchet (SPQR) 2025-06-10 16:54:30 -05:00
Jon Chambers
ca6e5fb0a8 Hide model validation methods in API docs 2025-06-06 16:19:48 -04:00
Jon Chambers
1a7a446150 Regenerate phone number identifiers when regenerating secondary table data 2025-06-05 15:12:33 -04:00
Ameya Lokare
981d929f50 Extend ChannelCircuitBreakerHandler with ChannelOutboundHandlerAdapter
instead of ChannelDuplexHandler
2025-06-05 12:00:21 -05:00
Ravi Khadiwala
4a3eb642c0 Remove unused S3Client 2025-06-05 11:49:03 -05:00
Ameya Lokare
a1b0c1a4aa Update to the latest version of the spam filter 2025-06-04 10:53:16 -07:00
Chris Eager
0f185a528d Add isUrgent tag to message delivery latency metrics 2025-06-04 10:51:05 -07:00
Ravi Khadiwala
aef7f3fef8 Avoid generating invalid deviceId in unit test 2025-06-04 12:49:23 -05:00
Ravi Khadiwala
1767586797 Add metrics for opk upload size 2025-06-04 10:46:11 -07:00
Ameya Lokare
60be6de9af Trivial: Add missing @Mutable annotation to setPublicKey 2025-06-03 18:13:39 -07:00
Jonathan Klabunde Tomer
2a7551cca5 support REST deprecation by platform for all requests with % rollout 2025-05-29 16:15:19 -07:00
Jonathan Klabunde Tomer
36439b5252 call ThreadLocalRandom.current at point of use only 2025-05-29 16:15:05 -07:00
ravi-signal
bbee80dbd0 Fix class cast exceptions with SchedulingUtil 2025-05-29 16:14:23 -07:00
Ravi Khadiwala
a7ea42adc3 Add a crawler to recalculate quota usage 2025-05-28 15:49:55 -05:00
Ravi Khadiwala
4dc3b19d2a Track backup metrics on refreshes 2025-05-28 15:28:55 -05:00
ravi-signal
030d8e8dd4 Reduce drift between tracked and actual backup usage 2025-05-28 15:25:32 -05:00
Chris Eager
401165d0d6 Convert unidentifiedDelivery.certificate configuration to byte[] 2025-05-27 14:55:14 -05:00
Chris Eager
ccb209ad37 Consolidate avatar deletion logic in ProfilesManager 2025-05-27 13:46:41 -05:00
Chris Eager
c1a66e0418 Delete avatars in ProfilesManager#deleteAll 2025-05-27 13:46:41 -05:00
Jon Chambers
8491d18413 Revert "Count API calls by authentication status"
This reverts commit 9b835633ab.
2025-05-27 13:51:17 -04:00
Jon Chambers
9b835633ab Count API calls by authentication status 2025-05-27 11:59:28 -04:00
Jon Chambers
fbbc4b8b27 Get integration test configuration directly from a GitHub Actions variable 2025-05-21 14:42:14 -04:00
Jonathan Klabunde Tomer
74ee1c8c4f Update to the latest version of the spam filter 2025-05-21 10:46:18 -07:00
Jonathan Klabunde Tomer
35604cf151 Simplify rate limiters by making them all dynamic 2025-05-21 10:29:26 -07:00
Ravi Khadiwala
aafcd63a9f Decrease the page size for OPK queries
A single element is almost always enough
2025-05-20 11:21:20 -04:00
Jon Chambers
43a534f05b Add a command for regenerating account constraint tables 2025-05-20 11:21:02 -04:00
Jon Chambers
9ec66dac7f Make getRegistrationId identity-type-aware 2025-05-14 14:39:11 -04:00
Jon Chambers
13fc0ffbca Assume that PNI registration IDs are always present on Device records 2025-05-14 14:39:11 -04:00
Jon Chambers
93ba6616d1 Perform device list validations in the scope of a pessimistic account lock 2025-05-14 14:39:11 -04:00
Jon Chambers
a4b98f38a6 Use a Callable for tasks performed within the scope of a pessimistic lock 2025-05-14 14:39:11 -04:00
Jon Chambers
b95d08aaea Drop PqKeysUtil 2025-05-14 14:39:11 -04:00
Jon Chambers
b400d49e77 Require PQ keys when changing numbers or distributing key material 2025-05-14 14:39:11 -04:00
Jon Chambers
e43487155f Remove commands for removing accounts/devices without PQ or PNI key material 2025-05-14 14:39:11 -04:00
Jon Chambers
dee3723d97 Remove an unused user-agent argument 2025-05-14 14:39:11 -04:00
Jon Chambers
b7e986f43c Add an integration test for changing phone numbers 2025-05-14 14:39:11 -04:00
Jon Chambers
664fb23e97 Resolve warnings/suggestions throughout AccountsTest 2025-05-14 11:30:59 -04:00
Chris Eager
714ef128a1 Compare using PNI in account reclamation 2025-05-13 16:41:42 -07:00
Ravi Khadiwala
7cf3fce624 Log unexpected account reclaim mismatches 2025-05-13 14:17:18 -05:00
ravi-signal
0cc5431867 Update noise-gRPC protocol errors 2025-05-13 14:16:23 -05:00
Ravi Khadiwala
b8d5b2c8ea Match account idle duration in RemoveExpiredBackupsCommand 2025-05-13 14:15:50 -05:00
Ravi Khadiwala
894ca6d290 remove ANDROID_SKIP_LOW_URGENCY_PUSH_EXPERIMENT 2025-05-13 13:59:28 -05:00
Ravi Khadiwala
847b25f695 Add experiment to coalesce android notifications 2025-05-13 13:59:28 -05:00
Ravi Khadiwala
703a05cb15 Support scheduling background FCMs 2025-05-13 13:59:28 -05:00
Jon Chambers
30c194c557 Exclude RateLimitExceededException from fail-open checks 2025-05-12 15:24:57 -07:00
Jonathan Klabunde Tomer
cc7b030a41 Send disconnection requests after non-API device unlinks 2025-05-06 13:36:41 -07:00
Jon Chambers
7a91c4d5b7 Correct metric names 2025-05-05 13:53:22 -04:00
Jon Chambers
287da6e7e3 Ignore already-locked accounts in PNI key cleanup operations 2025-05-05 13:53:22 -04:00
Katherine
7cf89764e7 Update FullTreeHead to use FullAuditorTreeHead 2025-05-05 10:44:57 -07:00
Jon Chambers
d316c72beb Add commands for removing accounts/devices without PNI key material 2025-05-05 12:10:47 -04:00
Katherine Yen
82d187cc45 Update key transparency protobufs 2025-05-02 10:40:53 -04:00
Jon Chambers
0c240d21d2 Update to the latest version of the spam filter 2025-05-02 10:40:07 -04:00
Jon Chambers
009252c831 Configure IP-keyed rate limiters to fail open 2025-05-02 10:30:29 -04:00
Jon Chambers
0c1146aaa5 Configure rate limiters with large initial capacities to fail open 2025-05-02 10:30:29 -04:00
Jon Chambers
4fd06594a0 Configure fast-regenerating rate limiters to fail open 2025-05-02 10:30:29 -04:00
Jon Chambers
4e175be88f Allow the "inbound message bytes" limiter to fail open 2025-05-02 10:30:29 -04:00
Jon Chambers
771a700acd Configure fail-open policy on individual rate limiters 2025-05-02 10:30:29 -04:00
Jon Chambers
e9bd5da2c3 Allow fail-open behavior for a wider range of exceptions 2025-05-02 10:30:29 -04:00
Jon Chambers
f64244f33a Remove an unused TURN rate limiter 2025-05-02 10:30:29 -04:00
Ravi Khadiwala
ed1417c3e3 Update to the latest version of the spam filter 2025-04-30 15:06:03 -05:00
ravi-signal
0398e02690 Add NoiseDirect framing protocol 2025-04-30 15:05:05 -05:00
Chris Eager
e285bf1a52 Fix test by using generic exists command 2025-04-29 13:05:10 -05:00
Ameya Lokare
2c9219d4f7 Update to the latest version of the spam filter 2025-04-29 10:57:05 -07:00
Jon Chambers
26b3b75054 Only fetch last-resort PQ keys for accounts with linked devices 2025-04-28 16:59:08 -04:00
Jon Chambers
cdb651b68f Add commands for removing devices without PQ keys 2025-04-28 15:45:27 -04:00
Ameya Lokare
91a36f4421 Update to the latest version of the spam filter 2025-04-28 11:59:43 -07:00
Jonathan Klabunde Tomer
21c1d71551 take advantage of list non-nullitude 2025-04-25 10:06:42 -05:00
Jonathan Klabunde Tomer
38befdb260 default lists to empty 2025-04-25 10:06:42 -05:00
Jonathan Klabunde Tomer
63c79173b2 limit prekey uploads to 100 2025-04-25 10:06:42 -05:00
Ameya Lokare
d2ad003891 Remove free memory and OS memory gauges 2025-04-25 10:05:29 -05:00
Chris Eager
eb89773819 Remove unused parameter 2025-04-25 10:05:18 -05:00
Chris Eager
403abd84f6 Run test action on pull_request events 2025-04-25 10:05:08 -05:00
Jon Chambers
f62f79c95c Add a counter for cases where clients use both an authenticated identity and UAK when fetching profiles 2025-04-24 11:47:43 -04:00
Jon Chambers
144c4c9223 Add a "sync" dimension to the "sent message" counter 2025-04-24 10:33:39 -05:00
Ravi Khadiwala
ab4fc4f459 Add skip low urgency push experiment 2025-04-24 10:32:46 -05:00
Jonathan Klabunde Tomer
51569ce0a5 Use cached partition topology for metrics/logs 2025-04-24 08:29:58 -07:00
Jon Chambers
f191c68efc Close remote connections only after all active server calls have completed 2025-04-22 17:00:48 -04:00
Jon Chambers
bb8ce6d981 Introduce ClosableEpoch 2025-04-22 17:00:48 -04:00
Katherine
e0ee75e0d0 Fix Daylight Savings bug in recommended notification time calculation 2025-04-22 16:56:10 -04:00
Jon Chambers
1ef3a230a1 Tag queue size distribution with client platform 2025-04-22 16:55:16 -04:00
Jon Chambers
b1805d4bf1 Add a "persisted bytes" counter 2025-04-22 16:55:16 -04:00
Jon Chambers
cac979c7fd Count individual persisted messages 2025-04-22 16:55:16 -04:00
Jon Chambers
4072dcdda5 Introduce DevicePlatformUtil 2025-04-22 16:55:16 -04:00
Jonathan Klabunde Tomer
ed382fff6d log slot number and shard host of message persister failures 2025-04-22 16:55:16 -04:00
Jon Chambers
23bb8277d5 Update to the latest version of the spam filter 2025-04-18 15:56:17 -04:00
Jon Chambers
8099d6465c Clarify guarantees around remote channnel/request attribute presence 2025-04-18 15:44:21 -04:00
Jon Chambers
28a0b9e84e Include a TURN credential TTL for clients in GetCallingRelaysResponse 2025-04-17 10:30:58 -04:00
Chris Eager
9287aaf7ce Add app info to Stripe API calls 2025-04-17 09:30:34 -05:00
Chris Eager
0585f862cb Add regression test for set profile badges calculation 2025-04-17 09:29:11 -05:00
Chris Eager
7cac6f6f72 Remove extraneous account fetch in POST /v1/donation/redeem-receipt 2025-04-17 09:28:57 -05:00
Jon Chambers
57be4d798b Add a counter for attempts to send empty message lists 2025-04-17 10:27:46 -04:00
Jon Chambers
05c74f1997 Simplify UserAgentUtil 2025-04-17 10:27:24 -04:00
Jon Chambers
f5e49b6db7 Convert UserAgent to a record 2025-04-15 14:58:09 -04:00
Jon Chambers
3c40e72d27 Fix registration ID map construction when changing numbers 2025-04-15 14:57:28 -04:00
Ravi Khadiwala
2f2ae7cec5 simplify story tag calculation 2025-04-11 14:04:09 -05:00
Chris Eager
b236b53dc3 set profile: move updated badge calculation into account updater lambda 2025-04-11 14:03:05 -05:00
Katherine
eb71e30046 Update to protobuf 4.x 2025-04-10 13:05:23 -04:00
Jon Chambers
aa5fd52302 Explicitly pass sync message sender device ID as an argument to sendMessage 2025-04-10 11:40:32 -04:00
Jon Chambers
d6bc2765b6 Close gRPC channels from a copied list to avoid concurrent modification issues 2025-04-09 21:54:18 -04:00
Jon Chambers
01258de560 Throw a MismatchedDevicesException for empty message lists to support iOS clients 2025-04-09 21:53:58 -04:00
Jon Chambers
3af2cc5c70 Add tests for spam-reporting token presence 2025-04-09 14:24:34 -04:00
Jon Chambers
2278842531 Add gRPC endpoints for sending messages from identified/authenticated senders 2025-04-09 14:24:34 -04:00
Jon Chambers
60ab00ecc6 Specify bounds for message timestamps 2025-04-09 14:24:20 -04:00
Jon Chambers
1fb6d23500 Allow range validators to accept 64-bit min/max values 2025-04-09 14:24:20 -04:00
Jon Chambers
8d8a2a5583 Extract common message-sending methods into a shared utility class 2025-04-08 17:39:45 -04:00
Jon Chambers
caa81b4885 Implement story sending via gRPC 2025-04-08 17:30:33 -04:00
Jon Chambers
37c4a0451a Simplify returning spam responses from gRPC 2025-04-08 17:30:33 -04:00
Jon Chambers
11df8fcc6c Add gRPC endpoints for sending unauthenticated (i.e. sealed-sender) messages 2025-04-08 17:30:33 -04:00
Jon Chambers
5a7f4d8381 Make the utility method for checking group send credentials blocking 2025-04-08 17:30:33 -04:00
Jon Chambers
1f1e4c72ec Add simple-grpc as a dependency/generator 2025-04-08 17:30:33 -04:00
Jon Chambers
02a5a6b55f Subdivide MessageController's message-sending methods into message-type-specific methods 2025-04-08 10:22:07 -04:00
Ameya Lokare
58ad647d29 Update to the latest version of the spam filter 2025-04-07 11:25:49 -07:00
Jon Chambers
099073356c Make destination service identifiers for spam checks non-optional 2025-04-07 11:12:07 -07:00
Chris Eager
37038c4a63 Keep a strong reference to expiration in Gauge 2025-04-07 13:10:41 -05:00
Jon Chambers
ffa98e5b34 Reduce and centralize message-sending metrics 2025-04-07 11:08:53 -04:00
Jon Chambers
6013d00654 Introduce a distinct UA for server-generated messages 2025-04-07 11:01:01 -04:00
Jon Chambers
c03d63acb8 Centralize message size validation in actual message-sending methods 2025-04-07 11:01:01 -04:00
Jon Chambers
c6689ca07a Internalize destination device list/registration ID checks in MessageSender 2025-04-07 09:15:39 -04:00
Chris Eager
1d0e2d29a7 Update javax.ws Javadoc references to jakarta.wx 2025-04-07 09:11:36 -04:00
Ameya Lokare
d83d826236 Use expire NX on cardinality estimator key 2025-04-07 09:11:24 -04:00
Chris Eager
2efe687b4b Add TlsCertificateExpirationUtil 2025-04-07 09:10:48 -04:00
Katherine Yen
7cabc8f328 Update to the latest version of the spam filter 2025-04-03 12:27:43 -04:00
Katherine
b57bdcaaea Dependency updates for Q2 2025 2025-04-03 12:05:51 -04:00
Chris Eager
041643783d Add libsignal tag to MetricsHttpChannelListener 2025-04-03 11:05:20 -05:00
Jon Chambers
ed75ef917a Update to the latest version of the spam filter 2025-04-02 13:27:49 -04:00
Jon Chambers
7ea0885474 Accommodate gRPC in the SpamChecker interface 2025-04-02 13:16:55 -04:00
Jon Chambers
488e7c4913 Update to the latest version of the spam filter 2025-04-02 09:00:35 -04:00
Jon Chambers
6af4d41322 Enumerate rate limit challenge options 2025-04-02 08:48:43 -04:00
Ravi Khadiwala
2b07a21477 Add some additional backup metrics 2025-04-02 08:48:18 -04:00
Jon Chambers
4a42ff562d Define authenticated and anonymous gRPC services for sending messages 2025-04-02 08:47:34 -04:00
Jon Chambers
d4031893cc Make raw User-Agent strings available to gRPC services 2025-04-01 09:11:37 -04:00
Ravi Khadiwala
c2e3ab832c enforce provisioning websocket timeouts 2025-03-26 16:28:51 -05:00
Ravi Khadiwala
8c2f3c839f Fix calculation of backup staleness metric 2025-03-26 16:28:38 -05:00
Ravi Khadiwala
df188e6f15 Document behavior for deletes of non-existing objects 2025-03-26 16:26:18 -05:00
Jon Chambers
12576daf1f Retire the (long-unused!) KEY_EXCHANGE envelope type 2025-03-26 11:32:23 -04:00
Chris Eager
46aefc0cbe Add retry configuration to RemoveExpiredUsernameHoldsCommand 2025-03-25 11:46:05 -05:00
Chris Eager
9d4f9b4c12 Add retry configuration to RemoveExpiredBackupsCommand 2025-03-25 11:46:05 -05:00
Chris Eager
3591e6bebd Add retry configuration to RemoveExpiredAccountsCommand 2025-03-25 11:46:05 -05:00
Chris Eager
e12ba6b15b Reduce page size in MessagesDynamoDb#mayHaveUrgentMessages 2025-03-25 11:45:47 -05:00
Chris Eager
744b05244d Add onErrorResume and retries to eligibility check in NotifyIdleDevicesCommand 2025-03-25 11:45:16 -05:00
Chris Eager
c22b8fafa6 Update AWS SDK to 2.31.7 2025-03-25 11:44:54 -05:00
Ameya Lokare
2194cf46e1 Remove jaxb and jaxb-api version pins 2025-03-24 16:37:34 -07:00
Jonathan Klabunde Tomer
1a46ac122a add some tags to ProfileController.versionNotFound 2025-03-24 14:57:29 -07:00
Ameya Lokare
55f85a81c6 Update to the latest version of the spam filter 2025-03-24 13:26:53 -07:00
Chris Eager
5a8cb8a312 Revert "Skip shared multi-recipient message payloads for small messages"
This reverts commit db2cd20dcb.
2025-03-24 14:40:12 -05:00
Jon Chambers
d641d1fc39 Update to the latest version of the spam filter 2025-03-24 11:17:49 -04:00
Jon Chambers
df56c65b54 Validate device message sizes when distributing PNI keys 2025-03-24 10:58:38 -04:00
Jon Chambers
1346fcb59e Require that incoming messages have content 2025-03-24 10:58:38 -04:00
Jon Chambers
50f681ffe8 Centralize message length validation 2025-03-24 10:58:38 -04:00
Jon Chambers
faef614d80 Parse message content as a byte array in request entities 2025-03-24 10:58:38 -04:00
Chris Eager
db2cd20dcb Skip shared multi-recipient message payloads for small messages 2025-03-21 16:51:21 -05:00
Chris Eager
9ef6024291 Update to the latest version of the spam filter 2025-03-20 10:44:47 -05:00
Jon Chambers
a643a6c0f0 Always record inbound message byte rate limit events 2025-03-20 10:44:10 -05:00
Chris Eager
82e21b0c21 Remove obsolete turn implementations 2025-03-20 10:38:14 -05:00
Chris Eager
50e298a4f4 Add counter for Redis events 2025-03-20 10:37:18 -05:00
Chris Eager
a0b9c40f6c Add JettySslHandshakeMetrics 2025-03-20 10:36:52 -05:00
Ravi Khadiwala
359cf02161 Add new attachmentBackfill capability 2025-03-19 16:03:40 -05:00
Chris Eager
6a1f01f876 Remove unused executor service 2025-03-13 14:49:04 -04:00
Jon Chambers
0595acc48f Add a counter for messages removed from the message cache 2025-03-13 14:48:34 -04:00
Jon Chambers
46ff8d51dc Switch to remote aggregation for persister distribution summaries 2025-03-13 14:48:34 -04:00
Jon Chambers
dcd80e11f4 Measure persisted message bytes 2025-03-13 14:48:34 -04:00
Jon Chambers
e1b1c7db8d Normalize persister metric names; make metrics static final 2025-03-13 14:48:34 -04:00
Jon Chambers
99041bc593 Use a dedicated status code (and message) for "reauthentication required" 2025-03-13 14:48:17 -04:00
Ravi Khadiwala
dbd14481ed Add a missing join to testTrimOnFullPrimaryQueue 2025-03-11 17:39:15 -05:00
ravi-signal
e3160bc717 Add a dedicated size estimation method to MessagesCache 2025-03-10 16:09:05 -05:00
Chris Eager
6798958650 Update autoPagingIterable() usage to remove null params 2025-03-08 09:08:27 -08:00
Chris Eager
3a90c572b4 Ensure customer ID matches in StripeManager#cancelAllActiveSubscriptions 2025-03-08 08:35:43 -08:00
ravi-signal
eab3c36d83 Dont discard ephemeral messages beyond what the persister consumes 2025-03-07 15:27:03 -06:00
Ravi Khadiwala
b7fee7b426 Allow cancellation of an expired purchaseToken 2025-03-07 15:26:33 -06:00
ravi-signal
469955aec9 Treat a 410 for an IAP token as not found 2025-03-07 15:24:33 -06:00
Chris Eager
d1c9dff2c5 Use WhisperServerService as base name for executor service metrics 2025-03-07 15:23:51 -06:00
Jonathan Klabunde Tomer
9491ebbe90 remove versionedExpirationTimer 2025-03-06 08:20:06 -08:00
Ravi Khadiwala
09b50383d7 Automatically trim primary queue when cache cannot be persisted 2025-03-05 15:56:00 -06:00
Jon Chambers
8517eef3fe Add a "critical" warning tier for primary devices missing PQ keys 2025-03-05 08:51:10 -05:00
Jon Chambers
8955e31a1e Configure an "idle primary device reminder" interceptor 2025-03-04 08:35:01 -05:00
Jon Chambers
f7a3971c64 Add an authentication interceptor that adds alert headers for idle primary devices 2025-03-04 08:35:01 -05:00
Jon Chambers
552079d3c2 Add an interceptor interface for WebSocket authentication 2025-03-04 08:35:01 -05:00
Chris Eager
59d984e25d Add pqKeyAvailable tag to GetKeys metric 2025-03-04 08:02:05 -05:00
Jonathan Klabunde Tomer
d17b9322b7 remove old AAACertificateServices root too 2025-03-04 08:01:53 -05:00
Jonathan Klabunde Tomer
12bc175776 remove expired GeoTrust certificate from push trust store 2025-03-04 08:01:53 -05:00
Ravi Khadiwala
376458efa8 remove extra FCM QUOTA_EXCEEDED logging 2025-03-04 08:01:38 -05:00
Adel Lahlou
886984861f remove performance based turn routing from CallRoutingControllerV2 2025-03-04 08:01:18 -05:00
Jon Chambers
b248b6bc12 Add the new APNs certificate to our trust store 2025-02-24 16:24:17 -05:00
ravi-signal
8d0d0d61f1 Add reregistration flag to account creation response 2025-02-21 14:13:04 -06:00
Ravi Khadiwala
26c348520f Revert "Add a 2-notification ttl=0 push notification experiment"
This reverts commit 4908a0aa9e.
2025-02-21 14:07:07 -06:00
Ravi Khadiwala
ec79386306 Only accept backup receipt redemption when account has a backup credential request 2025-02-21 14:06:51 -06:00
Katherine
093ac6fb16 Align UpdateValue proto with the KT service's definition 2025-02-21 12:00:33 -08:00
Ravi Khadiwala
68e2c511b7 Split up backup-id rotation rate limits 2025-02-18 10:52:43 -06:00
Ravi Khadiwala
47c82b42d9 Check push tokens for primary device platform tags 2025-02-18 10:49:49 -06:00
Ravi Khadiwala
e1a3b48c6e Only set X-Signal-Agent if present 2025-02-18 10:49:26 -06:00
Jon Chambers
b8b17ae473 Add methods for iterating over all account identifiers 2025-02-18 11:46:00 -05:00
Chris Eager
b203344ed4 Only log unexpected MRM retrieval errors 2025-02-18 10:45:04 -06:00
Chris Eager
6d30a45017 Discard stale messages from MessagesCache#getMessagesToPersist 2025-02-14 12:06:15 -06:00
Chris Eager
3f9863c441 Discard mrm messages that can never be sent 2025-02-14 12:06:15 -06:00
Chris Eager
47294ef6b8 Update README.md 2025-02-13 17:18:50 -06:00
Ravi Khadiwala
2356d7c629 Add primary platform tag to transfer archive metrics 2025-02-13 10:25:41 -06:00
ravi-signal
4908a0aa9e Add a 2-notification ttl=0 push notification experiment 2025-02-13 10:25:25 -06:00
Jon Chambers
6032764052 Validate parsed message size, not base64-encoded message size 2025-02-10 14:13:24 -08:00
Jonathan Klabunde Tomer
908a41814b correct some theoretical bugs in unresolved service ID collection 2025-02-10 14:13:11 -08:00
Jon Chambers
3ae145bd60 Add isStory and isSyncMessage dimensions to message size metrics 2025-02-10 09:08:40 -08:00
Jonathan Klabunde Tomer
b086a73353 Return 200 and unregistered recipient list for multi-recipient send with GSEs 2025-02-10 09:08:21 -08:00
Ravi Khadiwala
794e254d90 Add null check to FCM error log condition 2025-02-07 13:48:36 -06:00
Ravi Khadiwala
760c5737f9 Add field to RestoreAccountRequest for device transfer initialization 2025-02-07 13:43:57 -06:00
Jonathan Klabunde Tomer
ea17eee320 Make logging of message delivery loops configurable, default off 2025-02-07 13:43:28 -06:00
Adel Lahlou
09ce79bd43 Remove /v1/calling/relays API endpoint 2025-02-07 13:41:59 -06:00
Chris Eager
2dfd17af4a Add counter with WebSocket close codes 2025-02-07 13:41:37 -06:00
Ravi Khadiwala
a9975e524b Add logs to FCM quota failures 2025-02-07 13:41:14 -06:00
Jonathan Klabunde Tomer
5d062285c2 Filter to block old REST API for specified client versions 2025-02-05 12:26:47 -08:00
Chris Eager
e4b0f3ced5 Use HTTP status code if FCM error code is unavailable 2025-02-05 14:17:05 -06:00
Jon Chambers
6545bb9edb Update to the latest version of the spam filter 2025-01-31 12:58:16 -05:00
Jon Chambers
70ce6eff9e Include ephemeral flag in individual messages 2025-01-31 12:57:54 -05:00
Chris Eager
c84d96abee Remove deprecated svr3Credentials field 2025-01-31 11:57:15 -06:00
Chris Eager
09eb42e5c6 Add tag for requests made with libsignal 2025-01-31 11:56:56 -06:00
Chris Eager
06388b514c Add timeout to GitHub test action 2025-01-31 11:56:39 -06:00
Jon Chambers
7c17a4067c Update to the latest version of the spam filter 2025-01-31 10:34:14 -05:00
Jon Chambers
48ada8e8ca Clarify roles/responsibilities of components in the message-handling pathway 2025-01-31 10:24:50 -05:00
Ravi Khadiwala
282bcf6f34 Add persistent timer utility backed by redis 2025-01-29 11:32:13 -06:00
Ravi Khadiwala
1446d1acf8 Fix blocking call in waitForLinkedDevice 2025-01-29 11:32:13 -06:00
Ravi Khadiwala
aae94ffae3 Add a timer to waitForTransferArchive 2025-01-29 11:32:13 -06:00
Ameya Lokare
ebd906a45d Update to the latest version of the spam filter 2025-01-28 12:21:09 -08:00
Ravi Khadiwala
e30beb9c9f Remove ZeroTtlDevicePushNotificationExperiment 2025-01-27 16:24:50 -06:00
Ravi Khadiwala
5a2e297991 Add ZeroTtlDevicePushNotificationExperiment 2025-01-27 12:58:28 -06:00
adel-signal
ae1e7fbaa0 Return empty lists instead of null in GetCallingRelaysV2 2025-01-24 17:33:45 -05:00
adel-signal
7e616a4056 Update calling routing to return urls only, no instance IPs 2025-01-24 13:46:32 -05:00
Chris Eager
c9e192564c Add fail-fast HMAC init to RegistrationServiceClient constructor 2025-01-22 18:20:45 -06:00
Chris Eager
47550d48e7 Add collation key to registration service session creation rpc call 2025-01-22 17:43:53 -06:00
Chris Eager
5cc76f48aa Add CoinGecko to CurrencyConversionManager 2025-01-19 09:37:38 -06:00
Katherine
3ceaa8bd20 Remove authentication via query parameters for websocket upgrade requests 2025-01-15 14:06:46 -05:00
Ameya Lokare
790b9bbf01 Update to the latest version of the spam filter 2025-01-14 12:38:44 -08:00
ravi-signal
ee1016523f Add request information to createPayPalOneTimePayment failure log 2025-01-14 09:45:22 -05:00
Katherine
2f51778421 Use camel casing for entry position field on key transparency monitor requests 2025-01-14 09:45:08 -05:00
Katherine Yen
d3d68c2a60 Enforce one entry position per identifier in monitor request 2025-01-10 10:06:31 -05:00
Jon Chambers
0628b3e41c Update chat service dependencies 2025-01-10 10:04:40 -05:00
Jon Chambers
eac183495a Update deployment Docker image 2025-01-10 10:04:40 -05:00
Jon Chambers
0ae02da9be Update GitHub Actions versions 2025-01-10 10:04:40 -05:00
Jon Chambers
90fe634ddd Update to the latest version of Maven 2025-01-10 10:04:40 -05:00
Ravi Khadiwala
1cae841ed6 Remove deprecated /v1/accounts/turn 2025-01-10 10:04:25 -05:00
Ravi Khadiwala
a88560e557 Add gRPC backup services 2025-01-08 16:13:51 -06:00
Ravi Khadiwala
3ca9a66323 Stripe HTTP2 clients in CloudflareTurnCredentialsManager 2025-01-08 16:02:22 -06:00
Chris Eager
3a4a55c245 Reject old-format Benin numbers, which are now undeliverable 2025-01-07 12:16:44 -06:00
Jonathan Klabunde Tomer
f4a243861c count messages between 8KiB and 256KiB 2025-01-03 10:18:55 -05:00
Jon Chambers
68209f270e Correct message topology in test JSON 2025-01-03 10:18:24 -05:00
Jon Chambers
8af939d320 Validate message timestamps 2025-01-03 10:18:24 -05:00
Jon Chambers
16a50935ea Add (failing!) tests for message timestamp validation 2025-01-03 10:18:24 -05:00
Chris Eager
24ea10c451 Add documentation annotations to /v1/config 2025-01-02 17:07:38 -06:00
Chris Eager
95abda4870 Remove Response wrapper from /v2/directory/auth 2025-01-02 17:07:26 -06:00
Chris Eager
c8993c4da8 Add documentation for /v2/directory/auth 2025-01-02 13:12:38 -06:00
Chris Eager
d096798340 Add documentation to /v1/storage/auth 2025-01-02 13:12:38 -06:00
Chris Eager
f2f5e0e26f Add documentation annotation to /v2/accounts/phone_number_discoverability 2024-12-27 12:24:00 -08:00
Chris Eager
9121032114 FaultTolerantRedisClusterClientTest: increase blpop timeouts to make tests more reliable 2024-12-27 12:23:49 -08:00
Jonathan Klabunde Tomer
541bf968e5 remove support for GET /v1/accounts/me 2024-12-27 12:23:14 -08:00
Chris Eager
8c3ebdcbab Remove obsolete ArtController 2024-12-20 12:18:45 -06:00
Ameya Lokare
0593e9e89f Add @NotBlank to verificationToken in LinkDeviceRequest 2024-12-20 10:18:11 -08:00
Ameya Lokare
0d412c88fd OpenAPI spec for VerificationController endpoints 2024-12-20 10:17:03 -08:00
Chris Eager
8280106493 Remove unused SVR3 controller and client 2024-12-20 12:14:50 -06:00
Ravi Khadiwala
a3e106fe04 Fix flaky websocketTimeoutNoHeader test 2024-12-18 19:07:50 -06:00
Chris Eager
981a04f33b Update lettuce to 6.5.1.RELEASE 2024-12-18 18:52:00 -06:00
Ameya Lokare
26025e5abd Rename preRegistrationExperiments to e164Experiments 2024-12-18 18:49:34 -06:00
Jon Chambers
4839a5ba70 Enable/disable AWS SDK metrics via dynamic configuration 2024-12-18 18:49:05 -06:00
ravi-signal
85a1550485 Catch and log unexpected createPayPalOneTimePayment responses 2024-12-18 18:46:46 -06:00
ravi-signal
77658415b2 Handle stripe amount_too_large errors 2024-12-18 18:46:22 -06:00
ravi-signal
68f27be7cd Add opt-in timeouts to provisioning websocket 2024-12-18 18:45:53 -06:00
Ravi Khadiwala
6460327372 Return 409 when setting a payment method that hasn't finished initialization 2024-12-17 12:56:31 -05:00
Jon Chambers
a96c0ec7a3 Enqueue async operations from a dedicated thread 2024-12-17 12:54:41 -05:00
Ravi Khadiwala
33c0a27b85 Add current entitlements to whoami response 2024-12-12 12:56:44 -06:00
Ravi Khadiwala
d5b39cd496 Add utility method for creating AccountIdentityResponse 2024-12-12 12:40:43 -06:00
Ravi Khadiwala
5a35d69ed0 Fix 409/410 docs in /v1/messages/ 2024-12-12 12:36:54 -06:00
Ravi Khadiwala
e1e94a33e2 Stop writing non tag-set field 2024-12-12 12:36:37 -06:00
Ravi Khadiwala
18c9b177f3 Relax issuing constraints in IssuedReceiptManager 2024-12-10 11:47:44 -06:00
Ravi Khadiwala
1970741049 Remove IssuedReceiptMigrationCommand 2024-12-10 11:46:17 -06:00
Ravi Khadiwala
f16428ce2a Register IssuedReceiptMigrationCommand 2024-12-10 11:46:17 -06:00
Ravi Khadiwala
14427523ae Add a tag->tag-set migration command 2024-12-06 14:54:45 -06:00
Ravi Khadiwala
236b0496d3 Write a set field in IssuedReceiptsManager 2024-12-06 14:53:10 -06:00
Ravi Khadiwala
f68ddf66e9 Day align DeviceCheck backup vouchers 2024-12-06 14:52:50 -06:00
Jon Chambers
20685b6d69 Clear "canceled at" timestamp when setting a new subscrition ID 2024-12-06 15:51:57 -05:00
Jon Chambers
4988b4e0f5 Refresh accounts before returning device lists 2024-12-05 14:35:18 -05:00
Katherine
651e444875 Delete from SVR3 in account deletion flow 2024-12-05 11:36:33 -05:00
Jon Chambers
2093fed554 Drop the unused setUnauthenticatedDelivery endpoint 2024-12-03 14:06:05 -06:00
Ravi Khadiwala
cc7bb8b549 Map stripe idempotency errors to 400 2024-12-03 14:05:43 -06:00
Ravi Khadiwala
a99ac14c6a Write number->pni mapping before checking rrp table 2024-12-03 14:05:31 -06:00
Ravi Khadiwala
2c163352c3 Add DeviceCheck API for iOS Testflight backup enablement 2024-12-03 14:05:02 -06:00
Jon Chambers
fb6c4eca34 Retire RemoveE164RecentlyDeletedAccountsCommand 2024-12-02 11:17:03 -05:00
Jon Chambers
142e2cbe9d Update to the latest version of the spam filter 2024-12-02 10:59:51 -05:00
Jon Chambers
c70dd119d3 Remove AbstractDynamoDbStore as a base class from Accounts 2024-12-02 10:57:45 -05:00
Katherine
9e312cbdfa Normalize Benin phone numbers to the new format before sending to registration service 2024-12-02 10:57:27 -05:00
Jon Chambers
4d87b741cd Add a command to remove e164-associated "recently-deleted account" records 2024-11-27 12:29:42 -05:00
Jon Chambers
d08bc4c413 Write "recently deleted account" rows exclusively by PNI 2024-11-27 12:29:42 -05:00
Ravi Khadiwala
f5d3d1e65d Remove BackfillBeninPhoneNumberFormsCommand 2024-11-27 12:21:30 -05:00
Jon Chambers
916314233f Retire DeleteE164RegistrationRecoveryPasswordsCommand 2024-11-27 12:12:58 -05:00
Jonathan Klabunde Tomer
c7e6ee7297 Update to the latest version of the spam filter 2024-11-26 16:36:45 -08:00
Jonathan Klabunde Tomer
557a6ecd4f Read deleted accounts by PNI rather than e164 2024-11-26 16:35:40 -08:00
Ameya Lokare
0e04cac800 Crawler to backfill PNI records of alternate forms of existing phone numbers 2024-11-26 18:46:08 -05:00
Jon Chambers
1db9258d39 Add a command to remove e164-associated registration recovery passwords 2024-11-26 18:38:27 -05:00
Jon Chambers
2803c2acdb Write registration recovery passwords exclusively by PNI 2024-11-26 18:31:36 -05:00
Ravi Khadiwala
8be43566a4 Fix swagger oneof annotations 2024-11-26 17:40:05 -05:00
Ameya Lokare
d865cec2a4 Insert alternate forms of phone numbers -> PNI atomically 2024-11-26 17:39:53 -05:00
Jon Chambers
6f0370a073 Update to the latest version of the spam filter 2024-11-26 17:39:37 -05:00
Jon Chambers
5b9f8177f2 Read registration recovery passwords exclusively by PNI 2024-11-26 17:31:08 -05:00
Jon Chambers
6967e4e54b Retire MigrateRegistrationRecoveryPasswordsCommand 2024-11-26 17:23:49 -05:00
Jon Chambers
96fb0ac3ae Add a utility method for testing if a number begins with a decimal prefix 2024-11-26 14:13:06 -06:00
Jon Chambers
7201938793 Add a utility method for getting alternate forms of (Benin) phone numbers 2024-11-26 14:13:06 -06:00
Jonathan Klabunde Tomer
cdd6f78c73 Handle errors in deleted-accounts crawler 2024-11-25 21:22:33 -05:00
Jon Chambers
ab94d3045d Add a buffer/shuffle pair to better distribute load across shards 2024-11-25 20:41:25 -05:00
Jon Chambers
ff4e2bdfb7 Refresh registration recovery password expirations before retrying an insertion 2024-11-25 16:45:28 -05:00
Jonathan Klabunde Tomer
ffed19d198 Create deleted-accounts records keyed by both e164 and PNI 2024-11-25 12:42:16 -08:00
ravi-signal
49d6a5e32d Add error reporting to /v1/devices/transfer_archive 2024-11-25 12:41:51 -08:00
Jon Chambers
3ba7ba4f92 Remove abstract base class from RegistrationRecoveryPasswords 2024-11-25 15:27:30 -05:00
Jon Chambers
43ffc996db Use a segmented scan on a separate scheduler for registration recovery passwords 2024-11-25 14:18:56 -05:00
Jon Chambers
27f5f94c60 Retry registration recovery password migration attempts 2024-11-25 10:39:08 -05:00
Jonathan Klabunde Tomer
1d9734c824 MigrateRegistrationRecoveryPasswordsCommand: un-invert semantics of dryRun flag 2024-11-23 09:23:36 -05:00
Jon Chambers
717fb57a14 Add a command to migrate registration recovery passwords to PNI-associated records 2024-11-22 18:18:40 -05:00
Jon Chambers
af1d21c225 Add methods for migrating E164-mapped registration recovery passwords to PNI-mapped records 2024-11-22 18:18:40 -05:00
Jon Chambers
3c8b2a82a3 Store registration recovery passwords by both E164 and PNI 2024-11-22 18:18:40 -05:00
Jonathan Klabunde Tomer
0cb6f662c6 Handle multiple values with same ACI in deleted-accounts table
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2024-11-22 15:11:27 -08:00
Chris Eager
739ed56b4c Don’t use an existing record's number in AccountsManager re-registration handling 2024-11-22 17:10:08 -06:00
Chris Eager
9a19ef82fd Use pni in DynamoDB account put condition expression 2024-11-22 17:10:08 -06:00
Chris Eager
5627ed141b Update phone number/PNI constraint cancellation reason comment 2024-11-22 17:10:08 -06:00
Chris Eager
e4a2af67b1 Update AccountsManager#changeNumber validation to check for PNI equality 2024-11-22 17:06:42 -06:00
Chris Eager
cba56f3263 Add @E164 to ChangeNumberRequest 2024-11-22 17:05:07 -06:00
Chris Eager
1c3cf39b8a Remove String e164 from AccountLockManager 2024-11-22 17:04:32 -06:00
Jonathan Klabunde Tomer
6421438f64 Delete deleted-account records by PNI as well as e164 2024-11-22 15:36:15 -05:00
Jon Chambers
8a63682c16 Drop the unused PhoneNumberIdentifiers#getPhoneNumber method 2024-11-22 15:26:44 -05:00
Jon Chambers
13a8c6256d Delete registration recovery passwords by both phone number and PNI 2024-11-22 15:26:44 -05:00
Jon Chambers
8c9cc4cce5 Make PhoneNumberIdentifiers operations asynchronous 2024-11-22 15:26:44 -05:00
Jon Chambers
0023cb2521 Resolve minor warnings/suggestions 2024-11-22 15:10:00 -05:00
Jon Chambers
010ed77345 Use libphonenumber to get country codes 2024-11-22 15:10:00 -05:00
Jon Chambers
00c11f5dd0 Use a more conventional name for a static constant 2024-11-22 15:10:00 -05:00
Jon Chambers
637e424506 Remove an unused "get number prefix" method 2024-11-22 15:10:00 -05:00
Chris Eager
eb55b80bdc Add PNI support to AccountLockManager 2024-11-22 14:09:52 -06:00
Chris Eager
73812b06be Fix class names in logger initialization 2024-11-21 16:09:41 -06:00
Ravi Khadiwala
d135957f0d Cancel past_due subscriptions immediately 2024-11-21 16:09:17 -06:00
Chris Eager
815fd44ab3 Update to the latest version of the spam filter 2024-11-21 16:08:08 -06:00
Chris Eager
4e8a48ab3d Update MicrometerAwsSdkMetricPublisher maxConcurrency metric name 2024-11-21 16:07:24 -06:00
Chris Eager
ea75c39b58 Use MRM shared data views 2024-11-19 15:31:31 -06:00
Jon Chambers
085f013bf9 Update to libphonenumber 8.13.50 in preparation for Benin phone number format changes 2024-11-19 16:20:43 -05:00
Ameya Lokare
c6b8d890e5 Update to the latest version of the spam filter 2024-11-15 12:30:38 -08:00
Jon Chambers
66783c9381 Include new/previous push token types as dimensions on "account created" counter 2024-11-15 15:28:46 -05:00
Jon Chambers
b32e67ff9e Resolve minor warnings/suggestions in AccountsManager 2024-11-15 15:28:46 -05:00
Alex Konradi
ba55d6caeb Only vend new credential format
Remove the flag that controlled which format was requested, and always return
the new thing.
2024-11-15 12:27:56 -08:00
Ravi Khadiwala
1eba04d37b Fix incorrect update target 2024-11-15 14:11:55 -06:00
Chris Eager
ecbb2f1399 Updates for org.apache.commons.lang3.RandomStringUtils deprecations 2024-11-15 14:11:41 -06:00
Chris Eager
93f4a91ebf Remove some deprecated/unused code 2024-11-15 14:11:41 -06:00
Chris Eager
a1ac5bd74c Bind maven-dependency-plugin properties goal to the process-test-classes phase 2024-11-15 14:11:04 -06:00
Chris Eager
0ca9e973ad Add ephemeral tag to MRM error retrieval counter 2024-11-12 15:00:36 -06:00
Chris Eager
ee5df0e11c Always store and fetch shared MRM data 2024-11-12 10:57:30 -06:00
Chris Eager
d53a6e4c42 Add counter for skipped stale ephemeral MRM fetches 2024-11-12 10:56:12 -06:00
Chris Eager
744042e8c8 Address some warnings in maven execution 2024-11-12 10:55:10 -06:00
Jon Chambers
444f6ca826 Consolidate "closed connection" errors, but add an "exception" dimension 2024-11-12 11:54:47 -05:00
Jon Chambers
0ea13ec528 Handle static "closed" exceptions 2024-11-12 11:54:47 -05:00
Jon Chambers
6845ba9b90 Remove legacy DisconnectRequested protobuf entity 2024-11-12 11:54:36 -05:00
Jon Chambers
2f890f7bb3 Send push notifications if we receive a "new message" notification, but no listener is present 2024-11-12 09:34:59 -05:00
Jon Chambers
3fefb24d71 Retire the legacy disconnection request system 2024-11-12 09:09:05 -05:00
Chris Eager
d6f890c7b9 Use AmazonDynamoDBLocal instead of DynamoDBProxyServer 2024-11-11 16:50:47 -06:00
Chris Eager
59e0137816 Update to the latest version of the spam filter 2024-11-11 16:50:19 -06:00
Chris Eager
dd4bc23e4f Update to Dropwizard 4 2024-11-11 16:32:24 -06:00
Jon Chambers
09fd5e8819 Shift authority for disconnection requests to DisconnectionRequestManager 2024-11-11 16:27:38 -05:00
Ravi Khadiwala
81f3ba17c7 Wait for messages in waitForNewLinkedDevice 2024-11-11 14:46:48 -06:00
Jon Chambers
3288d3d538 Mirror disconnection requests to DisconnectionRequestManager 2024-11-11 15:38:07 -05:00
Jon Chambers
7e861f388f Introduce DisconnectionRequestManager 2024-11-11 15:38:07 -05:00
Jon Chambers
1323b42169 Rename ClientConnectionManager to GrpcClientConnectionManager 2024-11-11 15:38:07 -05:00
Jon Chambers
a843f1af6c Rename PubSubClientEventManager to WebSocketConnectionEventManager 2024-11-11 15:30:26 -05:00
Ameya Lokare
52b759c009 Remove CORS configuration 2024-11-11 12:22:18 -08:00
Jon Chambers
f2f5815316 Resolve additional merge-related naming conflicts 2024-11-11 11:58:06 -05:00
Jon Chambers
2da00e162a Resolve a merge-related naming conflict 2024-11-11 11:47:00 -05:00
Jon Chambers
e53a9f3f1a Close WebSockets "harshly" if a close frame write fails 2024-11-11 11:42:01 -05:00
Jon Chambers
767f6a90e0 Remove subscriptions if we get pub/sub events without a registered listener 2024-11-11 11:42:01 -05:00
Jon Chambers
562b495a18 Publish "messages persisted" events when unlocking queues after a persistence run 2024-11-11 11:41:29 -05:00
Jon Chambers
084607f359 Retire the explicit "handle new message available" system in favor of implicit presence-on-insert values 2024-11-11 11:41:29 -05:00
Jon Chambers
eeeb565313 Return destination client presence when inserting messages 2024-11-11 11:41:29 -05:00
Jon Chambers
1fa31b3974 Don't return message IDs from the "insert message" script 2024-11-11 11:41:29 -05:00
Jon Chambers
7158a504fa Remove an outdated doc comment reference to keyspace notifications 2024-11-08 15:55:02 -05:00
Ameya Lokare
3ccd7508ac Update to the latest version of the spam filter 2024-11-08 12:24:06 -08:00
Ravi Khadiwala
fc6075f19c Include media TTL in backup config 2024-11-08 14:21:39 -06:00
Ravi Khadiwala
776c147ea4 Tighten up Semaphore release in WebSocketConnection 2024-11-08 14:21:12 -06:00
Chris Eager
5d9641ae86 Add dynamic configuration to optionally use shared MRM data 2024-11-07 17:06:52 -06:00
Chris Eager
88a1f951c2 Update to the latest version of the spam filter 2024-11-07 16:53:15 -06:00
Chris Eager
1726a1d5f4 Remove extraneous plugin version declaration 2024-11-07 16:45:40 -06:00
Chris Eager
6c563a3f13 Update various dependencies
- aws.sdk2: 2.29.7
- braintree: 3.37.0
- commons-csv: 1.12.0
- commons-io: 2.17.0
- dropwizard: 3.0.10
- firebase-admin: 9.4.1
- google-androidpublisher: v3-rev20241016-2.0.0
- google-cloud-libraries: 26.50.0
- jackson: 2.18.1
- junit-pioneer: 2.23.0
- libphonenumber: 8.13.49
- micrometer: 1.13.6
- reactor: 2023.0.11
- storekit: 3.2.0
- exec-maven-plugin: 3.4.1
- jib-maven-plugin: 3.4.4
- maven-deploy-plugin: 3.1.3
- maven-install-plugin: 3.1.3
- maven-surefire-plugin: 3.5.2
- maven-failsafe-plugin: 3.5.2
- swagger-maven-plugin: 2.22.25
2024-11-07 16:34:42 -06:00
Jon Chambers
6a1f4906c5 Retire the legacy message availability system 2024-11-07 12:09:21 -05:00
Jon Chambers
ef716aacc2 Shift authority for message availability to the pub/sub event system 2024-11-07 11:55:47 -05:00
Ameya Lokare
c91242ed60 Update to the latest version of the spam filter 2024-11-07 08:32:15 -08:00
Jon Chambers
e536a40740 Publish "messages persisted" events 2024-11-07 11:25:48 -05:00
Jon Chambers
5aaf4cad20 Remove now-unused presence cluster configuration 2024-11-06 16:07:02 -05:00
Jon Chambers
1c167ec150 Retire the legacy client presence system 2024-11-06 14:57:27 -05:00
Chris Eager
9898e18ae2 Update test config for /v1/subscription/configuration 2024-11-06 14:43:39 -05:00
Chris Eager
b182c3d86d Update for KeyTransparencyQueryService.MonitorRequest changes 2024-11-06 14:43:18 -05:00
Jon Chambers
96a4d4c8ac Prepare to receive client events about persisted messages 2024-11-06 14:42:55 -05:00
Jon Chambers
9d19fc9ecc Shift authority to the new pub/sub client presence system 2024-11-06 12:10:44 -05:00
Jon Chambers
aad12670b2 Register WebSocket connections as pub/sub event listeners 2024-11-06 09:37:06 -05:00
Jon Chambers
7ca7fe7c13 Handle batches of commands in sharded circuit breakers 2024-11-06 09:36:08 -05:00
Jon Chambers
ca7e7c288e Format pub/sub hash tags consistently with message-related hash tags 2024-11-06 09:35:45 -05:00
Chris Eager
3a604464b5 Mark LevelConfiguration#name as deprecated 2024-11-06 09:35:19 -05:00
Jon Chambers
e277281d18 Update to the latest version of the spam filter 2024-11-06 09:32:57 -05:00
Ameya Lokare
3d339696dc Update to the latest version of the spam filter 2024-11-05 16:17:22 -08:00
Jon Chambers
3e36a49142 Reframe "connection ID" as "server ID" to avoid double-removing clients 2024-11-05 18:43:31 -05:00
Jon Chambers
d8f53954d0 Don't send CLIENT SETINFO commands when connecting to Redis instances 2024-11-05 17:29:31 -05:00
Jon Chambers
8c984cbf42 Introduce and evaluate a client presence manager based on sharded pub/sub 2024-11-05 15:51:29 -05:00
Jon Chambers
60cdcf5f0c Only respond to cluster toplogy events from the same cluster to which we're subscribed 2024-11-05 11:42:51 -05:00
Jon Chambers
5afcd634b6 Extend "long idle" device range to 60–75 days 2024-11-05 11:33:22 -05:00
Katherine Yen
00ca58ec13 Align FullTreeHead proto with KT's 2024-11-05 11:33:11 -05:00
Katherine Yen
bbb6d448db Standardize KT response field name to serializedResponse 2024-11-05 11:32:58 -05:00
adel-signal
45fad7a6a9 update e164 experiments to use account enrollment percentage 2024-11-04 15:00:25 -06:00
Katherine
7633a9b07a Replace TreeSearchResponse with CondensedTreeSearchResponse 2024-11-04 10:36:15 -05:00
Jon Chambers
00d0dba62c Don't retry pub/sub commands 2024-11-04 10:35:50 -05:00
Jon Chambers
c9a396b9e3 Deserialize null capabilities in Device entities as empty sets 2024-11-01 16:48:56 -04:00
Ravi Khadiwala
fc0a7b7657 401 instead of 403 on wrong backup auth credential type 2024-11-01 16:46:44 -04:00
Ameya Lokare
c1e870d8f5 Update to the latest version of the spam filter 2024-10-31 14:55:33 -07:00
Ameya Lokare
190f2a7fc2 Pass ACI to captcha checker 2024-10-31 14:24:43 -07:00
Jonathan Klabunde Tomer
ce0ccf4fd0 Update to the latest version of the spam filter 2024-10-31 11:27:38 -07:00
Jonathan Klabunde Tomer
0018e0bec6 remove appconfig in favor of S3ObjectMonitor 2024-10-31 11:25:45 -07:00
Jonathan Klabunde Tomer
63021e0ca3 Avoid annotation-driven parsing of device capabilities in DeviceController 2024-10-31 10:58:38 -04:00
Katherine Yen
bf741df38e Remove unnecessary NotNull annotation 2024-10-31 10:56:19 -04:00
Katherine
e627d4e2c4 Require some tree head sizes in key transparency search and monitor requests 2024-10-31 10:54:55 -04:00
Katherine
9b5a62e60f Remove FullTreeHead from TreeSearchResponse 2024-10-31 10:51:22 -04:00
Ravi Khadiwala
3bb1eab48c Fix casing in GET /v1/archives/auth 2024-10-30 17:07:43 -04:00
Jon Chambers
4af576668c Fix incorrect substitution of enumeration key names for capability names 2024-10-30 17:06:09 -04:00
Jon Chambers
f5a93574f6 Simplify capabilities representation in gRPC profile responses 2024-10-30 17:06:09 -04:00
Jon Chambers
74f8889bfa Simplify capability set serialization 2024-10-30 17:06:09 -04:00
Jon Chambers
a8da0f64ac Extract device capability enumeration translation to a utility class 2024-10-30 17:06:09 -04:00
Jon Chambers
44c3b046dd Add a new capability for storage service record key rotation 2024-10-30 13:04:10 -04:00
Ravi Khadiwala
f2cb04817b Let server generate copyToMedia IVs
We include the IV in the encrypted payload, so we can let the server
choose them instead of the client
2024-10-30 13:03:48 -04:00
Katherine
a5f60b1522 Use standard, unpadded base64 serialization/deserialization for commitmentIndex 2024-10-30 12:50:45 -04:00
Jon Chambers
0e3dccd9f6 Replace DeviceCapabilities entity with Set<DeviceCapability> 2024-10-30 12:46:20 -04:00
Jon Chambers
b21b50873f Add support for distinct media backup credentials
Co-authored-by: Ravi Khadiwala <ravi@signal.org>
2024-10-29 16:03:10 -04:00
Katherine
d335b7a033 Align KeyTransparencySearchResponse with distinguished counterpart and update DistinguishedResponse field names 2024-10-29 15:55:45 -04:00
Jon Chambers
f3b22e04e8 Allow primary devices to change names of linked devices 2024-10-29 09:52:38 -04:00
Katherine
712f3affd9 Update chat to send three search keys in one request to KT 2024-10-29 09:52:26 -04:00
Ameya Lokare
89292e238b Update to the latest version of the spam filter 2024-10-28 12:20:54 -07:00
Katherine
3287085ef9 Remove test references to legacy ratelimit HTTP status code 2024-10-28 15:20:08 -04:00
Ameya Lokare
17dfd914d5 Add maybe to optional argument name 2024-10-28 12:19:19 -07:00
Chris Eager
c65fe49983 Add commitment index to key transparency monitor requests 2024-10-28 14:18:41 -05:00
Chris Eager
6552d90dc9 Refactor request JSON creation in KeyTransparencyControllerTest 2024-10-28 14:18:41 -05:00
Chris Eager
d925e8af9e Update shape of KeyTransparencyMonitorRequest 2024-10-28 14:18:41 -05:00
Jon Chambers
2c0fc43137 Require that accounts be non-stale when getting/setting backup vouchers 2024-10-28 15:18:19 -04:00
Ameya Lokare
ccdbec088f Document that sendMessage can return 428 2024-10-28 12:18:02 -07:00
Jon Chambers
9822d17ab9 Fail "wait for X" futures if a Redis operation fails 2024-10-28 15:17:49 -04:00
Chris Eager
9573d9e385 Skip fetching MRM content for stale ephemeral messages 2024-10-24 11:49:33 -05:00
Chris Eager
155f3d6231 key transparency: add distinguished key endpoint 2024-10-24 11:49:20 -05:00
Chris Eager
1959ca2d96 Add client identity to key and certificate to KeyTransparencyServiceClient 2024-10-24 11:36:57 -05:00
Jon Chambers
324913d2da Add API endpoints for waiting for account restoration requests 2024-10-24 12:25:40 -04:00
Ameya Lokare
5c4cafcb6f Update to the latest version of the spam filter 2024-10-23 13:21:05 -07:00
Ameya Lokare
1ea8d69b40 Update captcha string in local testing doc 2024-10-23 09:56:41 -07:00
Katherine
013e45596e Update KT search requests to include a value and maybe an unidentified access key 2024-10-23 10:21:38 -04:00
Ameya Lokare
3fdb691702 Update to the latest version of the spam filter 2024-10-22 10:29:01 -07:00
Ameya Lokare
997129871c Lazily get captcha clients to avoid initialization issues 2024-10-22 09:13:00 -07:00
Ameya Lokare
39b1935350 Update to the latest version of the spam filter 2024-10-21 14:09:19 -07:00
Ameya Lokare
dbb9a8dcf6 Get captcha clients from spam-filter module 2024-10-21 13:32:32 -07:00
adel-signal
cacd4afbbb Add /v2/calling/relays
This supports returning IceServers from multiple providers at once
2024-10-21 13:59:29 -05:00
Jon Chambers
9c5877aa31 Resolve warnings/accept suggestions throughout AttachmentControllerV4Test 2024-10-21 14:26:28 -04:00
Jon Chambers
bda4788a34 Retire AttachmentControllerV2 2024-10-21 14:26:28 -04:00
Jon Chambers
5abfef50fc Avoid repeated constants, standardize test assertions, and add links to docs for PolicySigner 2024-10-21 14:26:04 -04:00
Jon Chambers
e3ee5c1f2e Use multiline strings in PostPolicyGenerator 2024-10-21 14:26:04 -04:00
Jon Chambers
e6eb702a88 Remove a couple unused imports/loggers from SubscriptionController 2024-10-21 13:18:00 -05:00
Jon Chambers
1447819198 Remove deprecated aci field from batch identity check elements 2024-10-21 13:17:52 -05:00
Jon Chambers
adf5795dff Remove unused members in ProfileController 2024-10-21 13:17:52 -05:00
Chris Eager
584fd06b88 Add metric for closed connection age to KeepAliveController 2024-10-21 13:17:21 -05:00
Chris Eager
1faa1a5abc Remove obsolete identity key null-check from /v1/certificate/delivery 2024-10-21 13:17:05 -05:00
Chris Eager
b5db8eba06 Remove unnecessary socket.setReuseAddress() in test extensions 2024-10-21 13:16:57 -05:00
Chris Eager
c6843c1eae Update to embedded-redis 0.9.1 2024-10-21 13:16:45 -05:00
Ameya Lokare
93b7fd589e Update to the latest version of the spam filter 2024-10-17 09:57:32 -07:00
Jon Chambers
eb80305f87 Retire AttachmentControllerV3 2024-10-17 12:48:01 -04:00
Jon Chambers
865e3c5bde Convert AccountBadge to a record 2024-10-17 12:47:50 -04:00
Chris Eager
c2270e57df Reduce log level for existing pending presence 2024-10-17 11:47:18 -05:00
Ameya Lokare
d48c031548 Pass in destination service identifier to spam filter 2024-10-17 09:11:35 -07:00
Chris Eager
830a07012b Subscribe to remote presence changes before setting the key 2024-10-15 16:54:52 -05:00
Chris Eager
46227295ff Add test for concurrent connections in ClientPresenceManager 2024-10-15 16:54:52 -05:00
Jon Chambers
73fb1fc2ed Add API endpoints for waiting for transfer archives 2024-10-15 12:13:47 -04:00
Jon Chambers
7ff48155d6 Add plumbing for a "wait for transfer archive" system 2024-10-15 12:13:47 -04:00
Jon Chambers
0adaa331a1 Extract RemoteAttachment to a top-level entity 2024-10-15 12:13:47 -04:00
Jon Chambers
30ec06ca76 Fix an incorrect response schema annotation 2024-10-15 12:13:47 -04:00
Jon Chambers
9b5c6e538b Convert DeviceResponse to a record and rename to LinkDeviceResponse 2024-10-15 12:13:29 -04:00
Jon Chambers
240a406964 Introduce EnumMapUtil 2024-10-10 13:29:20 -07:00
Jon Chambers
38d25f9a9b Resolve warnings and accept suggestions throughout DeviceControllerTest 2024-10-10 13:29:05 -07:00
Chris Eager
7b1b6fa1cf Update KeyTransparencyService.proto with MonitorKey.commitment_index 2024-10-10 10:12:21 -04:00
Chris Eager
a3d9af132f Use Mono#transformDeferred when applicable 2024-10-10 10:12:04 -04:00
Chris Eager
5d8b566a27 Remove unnecessary MessagesManager#clear with account’s PNI 2024-10-10 10:12:04 -04:00
Jon Chambers
8c30a359e7 Add API endpoints for waiting for newly-linked devices 2024-10-10 10:11:32 -04:00
Jon Chambers
087c2b61ee Update to the latest version of the spam filter 2024-10-09 09:37:07 -04:00
Jon Chambers
a9117010f9 Introduce FaultTolerantRedisClient 2024-10-09 09:22:10 -04:00
Ravi Khadiwala
9d980f36b0 Allow experiments to exclude by ACI 2024-10-08 15:02:58 -05:00
Katherine
93515e5a0f Retire paymentActivation capability 2024-10-08 16:01:09 -04:00
Jonathan Klabunde Tomer
d550c69f7f Enforce per-IP rate limits 2024-10-07 16:57:29 -05:00
adel-signal
087e192fac Update CallRoutingController to use e164, add UUID to e164 experiments 2024-10-07 16:56:45 -05:00
Jon Chambers
63e45563ec Discard the Redis-backed "used token" system 2024-10-07 17:06:08 -04:00
Jon Chambers
961d6d0a5c Update to the latest version of the spam filter 2024-10-07 16:31:51 -04:00
Jon Chambers
f7aacefc40 Manage device linking tokens transactionally 2024-10-07 16:26:11 -04:00
Ravi Khadiwala
42e920cd5c Add appstore subscriptions endpoint 2024-10-04 16:06:15 -05:00
ravi-signal
02ff3f2ff4 Add AppleAppStoreManager 2024-10-04 15:55:07 -05:00
Katherine
ca2845bcb0 Update key transparency gRPC client 2024-10-04 16:43:30 -04:00
Ravi Khadiwala
80cd5d9ccc Increase message cache TTL to 46 days 2024-10-02 14:05:40 -05:00
Ravi Khadiwala
0eb7db8de5 Increase the linked device idle timeout to 45 days 2024-10-02 14:05:40 -05:00
Jon Chambers
100955a7db Migrate WebSocket duration instrumentation to OpenWebSocketCounter 2024-10-02 14:02:03 -05:00
Chris Eager
68814813c3 Add timestamp header to all responses 2024-10-02 14:01:19 -05:00
Jon Chambers
39590f1b28 Update to the latest version of the spam filter 2024-10-01 14:27:06 -04:00
Jon Chambers
92698efd39 Remove machinery for setting/storing APNs VOIP tokens 2024-10-01 14:17:39 -04:00
Jon Chambers
b693cb98d0 Remove recurring background VOIP notification machinery 2024-10-01 14:17:39 -04:00
Jon Chambers
3ed142d0a9 Introduce OpenWebSocketCounter 2024-10-01 14:17:17 -04:00
Jon Chambers
581e61a85b Count open provisioning WebSockets 2024-10-01 14:17:17 -04:00
Jon Chambers
764b200289 Clarify open websocket gauge name 2024-10-01 14:17:17 -04:00
Jon Chambers
26503dffdf Use a consistent provisioning address 2024-10-01 13:34:37 -04:00
Jon Chambers
b284e95394 Clarify ProvisioningAddress field naming 2024-10-01 12:59:58 -04:00
Jon Chambers
b2211de8d8 Retire ProvisioningAddress and WebsocketAddress 2024-10-01 12:59:58 -04:00
Ameya Lokare
1bb0eb0e70 Integer comparison of captcha scores 2024-10-01 08:42:12 -07:00
Jon Chambers
0a1161048f Document ProvisioningController and ProvisioningConnectListener 2024-10-01 11:40:40 -04:00
Ameya Lokare
7a6ce00fed Add senderType tag to sendMessageLatency timer
This will allow us to differentiate between sealed vs unsealed sends
latency
2024-09-27 17:35:20 -04:00
Jon Chambers
c0aa9ced8d Temporarily disable AWS SDK instrumentation 2024-09-27 17:34:46 -04:00
Jon Chambers
6b36df3f8f Remove unused imports/loggers 2024-09-27 17:34:46 -04:00
Jon Chambers
df3caeb04a Expand the scope of the "notify idle devices" job to cover short-idle devices WITH messages 2024-09-27 16:30:51 -04:00
Jon Chambers
0e267509da Add a method for checking for persisted urgent messages 2024-09-27 16:30:51 -04:00
Jon Chambers
bbe41278ed Drop an unused hasCachedMessages method 2024-09-27 16:30:51 -04:00
Jon Chambers
d91a6b0c38 Retire the "notify idle devices with messages" experiment 2024-09-27 16:30:51 -04:00
Jon Chambers
694a93db6d Explicitly look for sha256sum inputs via stdin 2024-09-27 09:39:19 -04:00
Ameya Lokare
2f2dec87b1 Update to the latest version of the spam filter 2024-09-26 13:03:05 -07:00
Ameya Lokare
098288c290 Update to the latest version of the spam filter 2024-09-26 12:56:09 -07:00
Chris Eager
ab2e6bb9a3 Use Mono.share() for mrmMessageMono 2024-09-26 14:48:06 -05:00
Chris Eager
513f19370a Add Experiment#compareMonoResult 2024-09-26 14:48:06 -05:00
Chris Eager
e20a4c1f77 Refactor ProcessScheduledJobsServiceCommand to dispose of processing jobs on shutdown 2024-09-26 14:12:12 -05:00
Jon Chambers
946a486c4b Add metric publishers to DynamoDB clients 2024-09-26 15:00:10 -04:00
Jon Chambers
78b40397f9 Simplify metric publisher lifecycles 2024-09-26 15:00:10 -04:00
Jon Chambers
48e8d1c12f Add comments and constants to clarify the structure of metric collections 2024-09-26 15:00:10 -04:00
Jon Chambers
b115e95da4 Add an AWS SDK metric publisher 2024-09-26 15:00:10 -04:00
Chris Eager
ab0892cc41 Set executor shutdown time in ProcessScheduledJobsServiceCommand 2024-09-26 13:57:20 -05:00
Chris Eager
e25291c74c Move MRM content parsing outside of “withBinaryClusterReactive” 2024-09-26 13:57:02 -05:00
Ravi Khadiwala
0e552bd602 Cancel play subscriptions when replacing them 2024-09-26 13:56:10 -05:00
Ameya Lokare
e9b3e15556 Return report spam token from spam check instead of separate call 2024-09-26 11:53:40 -07:00
Ravi Khadiwala
237d0fd4e2 remove HTTP layer exceptions from Stripe/Braintree managers 2024-09-24 17:22:38 -05:00
Ravi Khadiwala
50bd30fb1f Fix cancelAtPeriodEnd calculation 2024-09-24 17:14:08 -05:00
Ravi Khadiwala
9fb4e2d272 set billingCycleAnchor in play billing responses 2024-09-24 17:14:08 -05:00
Jon Chambers
3e51366921 Narrow selection criterial for "notify idle devices with messages" experiment 2024-09-23 16:01:44 -04:00
Jonathan Klabunde Tomer
2d184b1ab6 return user's own username link handle in /whoami response 2024-09-23 12:46:10 -07:00
adel-signal
befcdf55fe Balance TURN routing options across datacenters 2024-09-23 12:43:43 -07:00
Ravi Khadiwala
ba12d39121 URLEncode shortCode paths 2024-09-23 14:43:15 -05:00
Ravi Khadiwala
bf0f553ced Don't return 204s as exceptions
The jersey LoggingExceptionMapper automatically adds an entity to
WebApplicationExceptions. Jersey's HTTP server later strips the body on
204 responses, but our custom WebSocketResourceProvider does not
2024-09-23 14:43:04 -05:00
Chris Eager
cd68a674bb Update protobuf-java to 3.25.5 2024-09-23 08:43:55 -05:00
Jon Chambers
315fc00eac Lazily evaluate jobs to give max concurrency meaningful "teeth" 2024-09-23 08:43:44 -05:00
Ameya Lokare
6142dcc7e6 Update to the latest version of the spam filter 2024-09-19 12:56:10 -07:00
Jonathan Klabunde Tomer
c47141ffda yield json errors from 500s in sendMultiRecipientMessage 2024-09-19 12:36:52 -07:00
Jon Chambers
6c8566db60 Align "allocate device" and "verify device" default rate limits 2024-09-19 14:59:06 -04:00
Jonathan Klabunde Tomer
aa60fae3b1 install RateLimitByIpFilter in soft-enforcement mode 2024-09-18 15:38:21 -07:00
Ravi Khadiwala
8cb9c60a3c Relax configured currency checks 2024-09-17 14:27:11 -05:00
Ameya Lokare
dd7a20a774 Log IOException from assessCaptcha 2024-09-17 12:01:13 -07:00
Chris Eager
374fe087bc Use destination service ID from the envelope when removing views from shared MRM data 2024-09-16 13:19:11 -05:00
Chris Eager
11691c3122 Update fields in HCaptchaResponse
This reverts commit 8d129b10ca.
2024-09-16 13:12:36 -05:00
Chris Eager
8d129b10ca Revert "Update field names in HCaptchaResponse"
This reverts commit 42d4574213.
2024-09-13 15:28:42 -05:00
Katherine
d6e03f50b9 Fix bug in ignoring svr errors in account deletion flow 2024-09-13 12:28:31 -04:00
Chris Eager
f60c9f2a15 Use destination service ID from the envelope when looking up in shared MRM data 2024-09-13 11:21:18 -05:00
Chris Eager
1c617284f3 Add MRM views experiment to MessagesCache.getMessagesToPersist() 2024-09-13 11:21:18 -05:00
Chris Eager
5bc6ff0e77 Add check for existing key to MessagesCacheInsertSharedMultiRecipientPayloadAndViewsScript 2024-09-13 11:21:18 -05:00
Jonathan Klabunde Tomer
020c21f4ef remove support for deprecated messages DynamoDB key schema 2024-09-13 09:18:41 -07:00
Katherine
bd57c1c7e7 Introduce configurable way to ignore SVR errors in the account deletion flow 2024-09-13 10:57:09 -04:00
Katherine Yen
f4b94a7a89 Update to the latest version of the spam filter 2024-09-13 10:55:35 -04:00
ravi-signal
b666b66160 Add missing @Valid annotations 2024-09-13 10:54:32 -04:00
ameya-signal
8bafb1a641 Update code style in MessageController.sendMessage() 2024-09-13 10:53:40 -04:00
Chris Eager
42d4574213 Update field names in HCaptchaResponse 2024-09-13 10:53:23 -04:00
Alex Konradi
9ef6f8aec9 Add a metric for requests of ZKC auth credentials 2024-09-13 10:53:04 -04:00
Jon Chambers
556eec649d Add platform and push token presence dimensions to account creation metrics 2024-09-06 12:37:23 -05:00
Ameya Lokare
e160025cfc Clean up OpenAPI annotations on PUT /v1/challenge
* Remove 413 response since we no longer send it
* Add missing 428 response for when invalid captcha is submitted
2024-09-06 10:36:02 -07:00
Chris Eager
0602149c52 Skip removeRecipientViewFromMrmData pipeline if there are no keys 2024-09-06 12:35:03 -05:00
Chris Eager
ad17c6e40d Wait for MRM experiment mono to complete before returning default message 2024-09-05 18:02:33 -05:00
Chris Eager
b95a766888 For ephemeral messages, remove recipient view from shared MRM data if recipient is offline 2024-09-05 18:02:33 -05:00
Chris Eager
a0770db179 Add timer to removeRecipientViewFromMrmData 2024-09-05 18:02:33 -05:00
Ameya Lokare
0601f6a35c Update to the latest version of the spam filter 2024-09-04 16:21:37 -07:00
Ameya Lokare
d6acfa56c2 Add platform tag to invalid HCaptcha reason metric 2024-09-04 15:28:36 -07:00
Chris Eager
11601fd091 Multi-recipient message views
This adds support for storing multi-recipient message payloads and recipient views in Redis, and only fanning out on delivery or persistence. Phase 1: confirm storage and retrieval correctness.
2024-09-04 13:58:20 -05:00
Chris Eager
d78c8370b6 Add timer to removeByGuid script 2024-09-03 12:33:39 -04:00
Jon Chambers
0b752409d5 Update to the latest version of the spam filter 2024-09-03 12:32:08 -04:00
Jon Chambers
46e0f5da74 Fix swapped placeholders in a log message 2024-08-30 17:01:11 -04:00
Jon Chambers
d835a2a450 Update to the latest version of the spam filter 2024-08-30 16:38:13 -04:00
Jon Chambers
f09cc03164 Detect message delivery loops 2024-08-30 16:27:21 -04:00
Jon Chambers
4c628b1cd9 Change the range for the "notify idle devices without messages" job to 30–45 days 2024-08-30 16:24:39 -04:00
Chris Eager
5122271750 Configure keyspace notifications in LocalFaultTolerantRedisClusterFactory 2024-08-30 13:00:31 -05:00
ravi-signal
7df978390f Add playProductId to backup level configuration 2024-08-30 12:59:09 -05:00
ravi-signal
564dba3053 Add playbilling endpoint to /v1/subscriptions 2024-08-30 12:50:18 -05:00
Ravi Khadiwala
3b4d445ca8 Don't try to update a cancelled stripe subscription 2024-08-30 12:47:26 -05:00
ravi-signal
176a15dace Add GooglePlayBillingManager 2024-08-28 14:22:37 -05:00
Jonathan Klabunde Tomer
9249cf240e Update to the latest version of the spam filter 2024-08-21 14:53:28 -07:00
Jonathan Klabunde Tomer
a049eda7e6 remove configuration for no-longer-used "metrics" redis cluster 2024-08-21 14:49:05 -07:00
Katherine
9eafa118d5 Return key transparency protobufs encoded as base64 strings 2024-08-21 17:08:06 -04:00
Katherine
8699d94de6 Give mock devices unique IDs when testing whether a capability is supported 2024-08-21 11:40:02 -04:00
Katherine
4c0a5ac3b2 Add versionedExpirationTimer capability 2024-08-21 11:39:43 -04:00
Ameya Lokare
fa51793379 Update to the latest version of the spam filter 2024-08-20 10:49:25 -07:00
Ameya Lokare
8b99df3169 Update to the latest version of the spam filter 2024-08-19 11:43:50 -07:00
Ameya Lokare
7cbbf73cc9 Add registration recovery checker 2024-08-19 11:43:19 -07:00
Jon Chambers
0b1ec1e50b Make push notification senders available via CommandDependencies 2024-08-16 16:52:07 -04:00
Jon Chambers
2744d33ef8 Update to the latest version of the spam filter 2024-08-16 16:17:55 -04:00
Jon Chambers
659ac2c107 Generalize push notification scheduler and add support for delayed "new messages" notifications 2024-08-16 16:16:55 -04:00
Jon Chambers
5892dc71fa Allow linked devices to unlink themselves via the gRPC API 2024-08-16 16:14:13 -04:00
Jon Chambers
fc3e547dce Remove an obsolete client version check when changing phone numbers 2024-08-16 16:14:00 -04:00
Ravi Khadiwala
e4f9f949f0 Serialize subscription errors as json 2024-08-15 18:30:48 -05:00
Fedor Indutny
7605462d48 Allow device to unlink itself 2024-08-15 13:54:07 -05:00
Ravi Khadiwala
fd10b9723d Add source length validation on backup media copy 2024-08-15 13:52:50 -05:00
Ravi Khadiwala
6cdfb7ab63 Reject authenticated one-time donation requests 2024-08-15 13:51:55 -05:00
Ravi Khadiwala
e5fdab1bc8 Return 400 if a client specifies paypal where it's not supported 2024-08-15 13:49:25 -05:00
Katherine
2aa1eee29d Align chat endpoints with "distinguished key" changes in key transparency service 2024-08-15 14:35:15 -04:00
Ravi Khadiwala
97e566d470 Move common subscription management out of controller 2024-08-15 13:29:26 -05:00
ravi-signal
a8eaf2d0ad Group one-time donation methods together 2024-08-15 13:25:09 -05:00
Ravi Khadiwala
b5f9564e13 Validate that sourceAttachments are valid base64 strings 2024-08-14 17:36:30 -05:00
Jon Chambers
7e353f8ea0 Update to the latest version of the spam filter 2024-08-14 12:52:57 -04:00
Jon Chambers
0075e94a42 Rename AuthenticatedAccount to AuthenticatedDevice 2024-08-14 12:44:48 -04:00
Katherine
1ea9e38fea Use a separate virtual executor instead of the one used for async jersey tasks 2024-08-14 12:28:16 -04:00
Jon Chambers
3b405a53d0 Move "push notifications on close" logic to WebSocketConnection 2024-08-14 12:24:49 -04:00
Katherine
84c329e911 Key transparency search and monitor endpoints 2024-08-12 13:14:42 -07:00
Jonathan Klabunde Tomer
4349ceaf0e Update to the latest version of the spam filter 2024-08-12 11:49:18 -07:00
Ravi Khadiwala
acdf37561f Count requests that supply auth to boost endpoints 2024-08-12 13:40:58 -05:00
Jon Chambers
9128d4cc49 Correct a counter name 2024-08-12 14:39:31 -04:00
Jon Chambers
206e97d374 Use queries instead of scans when fetching experiment samples (again) 2024-08-12 14:39:22 -04:00
Jonathan Klabunde Tomer
f682af2fe0 build a multi-architecture docker manifest list 2024-08-12 11:38:59 -07:00
Jon Chambers
ecf7e60d98 Add an experiment for sending push notifications to idle devices that DO have pending messages 2024-08-07 16:41:19 -04:00
Ravi Khadiwala
68ddc070ca Count old version authentication tokens 2024-08-07 15:41:08 -05:00
Ameya Lokare
d661da8d7e Update to the latest version of the spam filter 2024-08-06 15:37:09 -07:00
Jonathan Klabunde Tomer
5d2e8cb000 update to libsignal 0.54.2 2024-08-06 14:18:14 -07:00
Ravi Khadiwala
096bb8e6e5 Remove deprecated SubscriptionController methods 2024-08-06 16:09:44 -05:00
Jon Chambers
1af8bb494e Generalize "is idle?" check in idle device notification scheduler 2024-08-05 15:19:39 -04:00
Jon Chambers
46d04d9d1a Add a command to schedule notifications for idle devices without messages 2024-08-05 15:19:39 -04:00
Jon Chambers
c0ca4ffbcc Retire "notify idle devices without messages" push notification experiment 2024-08-05 15:19:39 -04:00
Katherine Yen
8720b6db95 Update to the latest version of the spam filter 2024-08-05 12:11:40 -07:00
Jon Chambers
8c61d45206 Tune the "finish push notification experiment" command 2024-08-05 15:02:24 -04:00
Katherine
0e4625ef88 Migrate to 429 for all ratelimit responses 2024-08-05 12:02:11 -07:00
Ravi Khadiwala
10d559bbb5 Return backup info at /v1/subscription/configuration
- Return the free tier media duration and storage allowance for backups
- Add openapi annotations
- Update default media storage allowance
2024-08-02 14:08:00 -05:00
Jonathan Klabunde Tomer
65b2892de5 Simplify unlink-device-on-full-DB process 2024-08-02 12:03:43 -07:00
Chris Eager
6fa6c3c81c Ensure multi-recipient messages are addressed to ACI service IDs 2024-08-01 12:31:27 -05:00
Chris Eager
e4ffc932a9 Check for IdentityType.PNI in OptionalAccess#verify 2024-08-01 12:31:27 -05:00
Ameya Lokare
8afc0e6ab2 Update to the latest version of the spam filter 2024-07-31 12:34:54 -07:00
Jon Chambers
822092044b Add a method to check for the presence of persisted messages, skipping the cache 2024-07-31 10:50:47 -04:00
Jon Chambers
f1c153f39f Log max concurrency when starting/finishing experiments 2024-07-31 10:50:31 -04:00
Jon Chambers
7e62dc64dc Replace filterWhen with flatMap 2024-07-31 10:50:31 -04:00
Jon Chambers
2104a60703 Also check legacy parittion keys for message presence 2024-07-31 10:50:19 -04:00
Jon Chambers
97785fa570 Remove unused metrics and arguments 2024-07-31 10:50:19 -04:00
Chris Eager
9341fe9584 Add endpoint tag and story to auth type tag values 2024-07-31 09:38:14 -05:00
Ravi Khadiwala
3a582721cf Throw error for oversized inbound noise messages 2024-07-30 11:25:09 -05:00
Ravi Khadiwala
3d96d73169 Break up large outbound noise messages 2024-07-30 11:25:09 -05:00
Ravi Khadiwala
542422b7b8 Replace XX/NX handshakes with IK/NK 2024-07-30 11:25:09 -05:00
Jon Chambers
c835d85256 Drop opentest4j from dependency management 2024-07-30 12:24:26 -04:00
Jon Chambers
56ada7f0e9 Add a "dry run" mode for the "start push notification experiment" command 2024-07-30 12:19:00 -04:00
Jon Chambers
56fdebde75 Check for cached/persisted messages in parallel 2024-07-30 12:18:46 -04:00
Jon Chambers
4ee67064bb Remove a pair of unused methods 2024-07-30 12:18:34 -04:00
Jon Chambers
045ec9689d Introduce a job scheduler and experiment for sending notifications to idle devices 2024-07-29 11:16:53 -04:00
Jon Chambers
4ebad2c473 Add a framework for running experiments to improve push notification reliability 2024-07-25 11:36:05 -04:00
Jonathan Klabunde Tomer
1fe6dac760 read old new key and new new key 2024-07-24 07:44:44 -07:00
Jonathan Klabunde Tomer
f12a6ff73f Remove migration paths for lazy message deletion 2024-07-23 14:07:19 -07:00
Chris Eager
6eed458ceb Use server timestamp for delivery duration metrics 2024-07-18 12:22:59 -05:00
Jon Chambers
54fb0a6acb Add a general job scheduler 2024-07-18 13:22:31 -04:00
Jon Chambers
5147d9cb6d Mark old attachment endpoints as @Deprecated 2024-07-17 15:01:44 -04:00
Jon Chambers
37369929f3 Retire PushLatencyManager 2024-07-17 15:01:23 -04:00
Jon Chambers
4f10014902 Add an "is primary" dimension to message latency measurements 2024-07-17 15:01:08 -04:00
Ravi Khadiwala
0ef3e00ba7 Use non-legacy rate limit error in BackupAuthManager 2024-07-11 16:39:11 -05:00
Chris Eager
2408590430 Update GitHub Actions 2024-07-11 16:38:15 -05:00
Chris Eager
b7f4fe4d73 Update various dependencies 2024-07-11 16:38:15 -05:00
Chris Eager
b811492acd Update maven-wrapper.properties 2024-07-11 16:38:15 -05:00
Chris Eager
a63e0e0390 Remove unused RateLimiters 2024-07-11 16:37:55 -05:00
Jon Chambers
5e8a0b2cfa Introduce a utility class for finding reasonable times to send push notifications 2024-07-11 17:36:54 -04:00
Chris Eager
eac75aad03 Add distribution of multi-recipient message recipient counts 2024-07-09 15:04:47 -05:00
Chris Eager
b05fbc2102 Add metric for group send token adoption 2024-07-09 15:04:12 -05:00
Jon Chambers
6d166fdfc5 Return futures from "send push notification" operations 2024-07-08 15:36:17 -04:00
Ravi Khadiwala
2e36673702 Add warn log if changeNumber messages fail 2024-07-08 15:28:08 -04:00
ravi-signal
0c81ffe8b7 Count unregistered APNS tokens with a recent update 2024-07-08 15:27:48 -04:00
Jon Chambers
02b9ceb4c7 Discard APNs tokens if the APNs server reports that the token is expired 2024-07-08 15:27:23 -04:00
Chris Eager
775889c0b6 Remove deprecated PUT /v2/keys/signed endpoint 2024-07-08 15:26:26 -04:00
Chris Eager
98f2cdaf5a Reduce DynamoDB test client API timeouts 2024-07-08 15:26:11 -04:00
Jon Chambers
ff5cc3cb4f Avoid duplicate metric registry configuration in commands 2024-07-08 15:25:16 -04:00
Jon Chambers
ebecb1caec Throw an exception if metric registries are configured more than once 2024-07-08 15:25:16 -04:00
Jon Chambers
73e0aea85c Retire Device#hasMessageDeliveryChannel() 2024-06-26 14:46:39 -04:00
Ameya Lokare
1a09f5807b Update to the latest version of the spam filter 2024-06-26 11:45:58 -07:00
Ameya Lokare
ec009a2bba Pass AccountAndAuthenticatedDeviceHolder to spam filter 2024-06-26 11:32:49 -07:00
Jon Chambers
f52c40a492 Update to the latest version of the spam filter 2024-06-25 11:57:37 -04:00
Jon Chambers
1959c059ed Rename AuthEnablementRefreshRequirementProvider to LinkedDeviceRefreshRequirementProvider 2024-06-25 11:57:25 -04:00
Jon Chambers
2d1610b075 Stop monitoring device "enabled" state changes from auth enablement refresh requirement provider
Device enabled states no longer affect anything at an authentication level
2024-06-25 11:57:25 -04:00
Jon Chambers
2f76738b50 Expire APNs tokens if they haven't been updated since the expiration timestamp 2024-06-25 11:53:23 -04:00
Jon Chambers
1cf174a613 Include "token invalidation timestamp" in push notification responses 2024-06-25 11:53:23 -04:00
Jon Chambers
d743454d07 Store updated APNs tokens unconditionally 2024-06-25 11:53:23 -04:00
Jon Chambers
1cd16eaa08 Delete FCM tokens immediately if FCM reports that they're no longer active 2024-06-25 11:53:23 -04:00
Jon Chambers
90e622b307 Require that message bundles include all linked devices 2024-06-25 11:51:09 -04:00
Jon Chambers
cb5cd64c05 Gracefully handle NotPushRegisteredException 2024-06-25 11:23:16 -04:00
Jon Chambers
2619569549 Update to the latest version of the spam filter 2024-06-25 09:57:32 -04:00
Jon Chambers
d306cafbcc Allow, but do not require, message delivery to devices without active delivery channels 2024-06-25 09:53:31 -04:00
Jon Chambers
f5ce34fb69 Update to the latest version of the spam filter 2024-06-24 15:41:11 -04:00
Jon Chambers
dbeba4f173 Reduce "delete after idle" time for accounts from 180 days to 120 days 2024-06-24 15:20:35 -04:00
Jon Chambers
86f83635bc Document a missing "capability downgrade" response 2024-06-24 15:20:06 -04:00
Jon Chambers
fceda00d83 Consider all device when checking device capabilities 2024-06-24 15:20:06 -04:00
Jon Chambers
9b7af00cf5 Add methods to test whether a device may have unread messages 2024-06-24 15:19:48 -04:00
Chris Eager
fa1281ae86 Update maven plugin versions 2024-06-24 14:16:05 -05:00
Chris Eager
f5de4d7b71 Update various library dependencies 2024-06-24 14:16:05 -05:00
Chris Eager
1134df88e2 Remove obsolete "recaptcha" 2024-06-24 14:15:27 -05:00
ravi-signal
4aadabfac0 Make copy/delete streaming friendly 2024-06-20 16:00:09 -05:00
Jon Chambers
c27898a993 Update to the latest version of the spam filter 2024-06-20 11:36:00 -04:00
Ravi Khadiwala
daa897db93 Add some extra validation in integration tests 2024-06-18 15:29:58 -05:00
Jon Chambers
7a907bb44d Update to the latest version of the spam filter 2024-06-14 15:01:37 -04:00
Jon Chambers
d7cb219577 Update to the latest version of the spam filter 2024-06-14 11:03:42 -04:00
Jon Chambers
b28f8b0e7f Update to the latest version of the spam filter 2024-06-14 10:34:37 -04:00
Jon Chambers
51721dde50 Update to the latest version of the spam filter 2024-06-13 14:40:35 -04:00
Jon Chambers
09547ba788 Update to the latest version of the spam filter 2024-06-13 10:39:11 -04:00
Jon Chambers
3dc8acc385 Update to the latest version of the spam filter 2024-06-13 09:10:30 -04:00
Katherine
0414da8c32 Add delete sync capability 2024-06-12 13:54:06 -04:00
Jon Chambers
155450380e Fix weird indentation 2024-06-12 13:53:48 -04:00
Jon Chambers
09bc4ef1d6 Measure sizes of group messages 2024-06-12 13:53:48 -04:00
Jon Chambers
3aa4d8713c Switch to a remote-aggregated distribution for message size metrics 2024-06-12 13:53:48 -04:00
Jon Chambers
5fc926271f Remove a metrics tag for "has spam reporting token" 2024-06-12 13:53:33 -04:00
Katherine
f435b612c9 Remove stale pni, pnp, and giftBadges capabilities 2024-06-12 13:42:18 -04:00
Jon Chambers
5b78c0d3e0 Update to the latest version of the spam filter 2024-06-10 17:21:04 -04:00
Jon Chambers
6a14bf70e0 Extend push token removal grace period from 3 days to 14 2024-06-10 17:16:49 -04:00
Jon Chambers
138b368951 Add an explicit "incorrect password" authentication failure reason 2024-06-10 17:16:32 -04:00
Jon Chambers
0871d6ebc1 Add a log filter for spurious warnings about unsupported channel options 2024-06-10 17:16:10 -04:00
Jon Chambers
ad5ef76e8e Update to the latest version of the spam filter 2024-06-10 11:20:52 -04:00
Jon Chambers
2f55747601 Remove expiration check from Device#isEnabled() 2024-06-07 10:39:11 -07:00
Jonathan Klabunde Tomer
b376458963 include deleted-messages counter on both message deletion paths 2024-06-07 10:24:42 -07:00
Jon Chambers
64ac22a918 Resolve warnings/suggestions throughout MessageControllerTest 2024-06-06 14:11:44 -04:00
Alan Liu
ffb81e4ff7 Retrieve Cloudflare Turn Credentials from Cloudflare 2024-06-05 12:03:40 -04:00
Jonathan Klabunde Tomer
01743e5c88 Delete messages lazily on account and device deletion to prevent timeouts when deleting accounts/devices with large queues 2024-06-04 12:16:43 -07:00
Ameya Lokare
4ef6266e8f Update spam-filter to the latest version 2024-06-03 15:33:25 -07:00
Ameya Lokare
478a8362b8 Update to the latest version of the spam filter 2024-06-03 12:17:02 -07:00
ravi-signal
afa1899dc9 Add a require.proto presence annotation 2024-06-03 14:07:02 -05:00
Ravi Khadiwala
cea2abcf6e Fix some accidentally sync async methods 2024-06-03 14:07:02 -05:00
Jonathan Klabunde Tomer
c7d1ad56ff support local-filesystem-based dynamic config for tests 2024-05-31 15:25:47 -07:00
ravi-signal
a5f490cc53 return nonzero exit code when check-dynamic-config fails 2024-05-30 16:21:51 -05:00
ravi-signal
abe29fa6ee document gRPC error strategy 2024-05-30 16:20:34 -05:00
Chris Eager
f6d1e566e7 Add validation annotations to DynamicExperimentEnrollmentConfiguration 2024-05-24 09:11:48 -04:00
Jon Chambers
9ec4f0b2f5 Gracefully handle proxy protocol messages at the beginning of TCP connections 2024-05-24 09:11:19 -04:00
Ameya Lokare
1678045ce4 Update to the latest version of the spam filter 2024-05-23 10:29:53 -07:00
Chris Eager
7286e724dc Add SIGNAL_SERVER_CONFIG override to LocalWhisperServerService 2024-05-23 12:08:48 -05:00
Jon Chambers
e59a1e9efd Add support for TLS 1.2 for the benefit of load balancers performing health checks 2024-05-23 08:31:23 -04:00
Jon Chambers
097bedcb9b Add a health check handler to the Noise-over-WebSocket pipeline 2024-05-22 14:46:05 -04:00
Jon Chambers
907ff89011 Use complete certificate chains from the TLS keystore 2024-05-22 12:38:29 -04:00
Chris Eager
08faa0c009 Only register metrics for connections to upstream nodes 2024-05-22 11:17:52 -05:00
Ravi Khadiwala
dd4759487b Specify AuthCheckResponseV3 serialization 2024-05-22 09:42:05 -04:00
Jon Chambers
7980da9ce5 Set client public keys in the scope of a pessimistic account lock 2024-05-22 09:40:48 -04:00
Jon Chambers
0e43524dac Remove client public keys when deleting accounts/devices 2024-05-22 09:40:48 -04:00
Jon Chambers
c5c5f642e8 Configure and instantiate a Noise-over-WebSocket tunnel 2024-05-22 09:37:25 -04:00
Jon Chambers
e096c608ee Make Noise-over-WebSocket component names more consistent 2024-05-22 09:37:25 -04:00
Jon Chambers
9a2bfe1180 Add a plaintext mode to the Noise-over-WebSocket server for local testing 2024-05-22 09:25:28 -04:00
Ravi Khadiwala
9e36cabef0 Update to the latest version of the spam filter 2024-05-17 11:20:27 -05:00
ravi-signal
ce1c5be940 Add svr3 share-set store/retrieve 2024-05-17 10:45:18 -05:00
Ravi Khadiwala
1182d159aa Move command boilerplate into a base class 2024-05-17 10:44:58 -05:00
Ravi Khadiwala
7d95926f02 Add a crawler for backup usage metrics 2024-05-17 10:38:00 -05:00
Jon Chambers
101ecf342f Remove now-unused rate limiters 2024-05-16 16:56:42 -05:00
Jon Chambers
4efba94662 Add an API endpoint for storing public keys 2024-05-16 17:53:16 -04:00
Ravi Khadiwala
1855d661e8 Add maximum length to ProvisioningMessage 2024-05-16 16:47:47 -05:00
Ravi Khadiwala
438abc4cf9 Remove unused entity 2024-05-16 16:47:47 -05:00
Ravi Khadiwala
40639f70f4 Fix flaky MessageMetricsTest
Make the MeterRegistry in MessageMetrics configurable
2024-05-16 13:39:17 -05:00
Ravi Khadiwala
a80c020146 Remove AssignUsernameCommand 2024-05-13 16:43:02 -05:00
Ravi Khadiwala
2ce3270d21 Update docker container image 2024-05-13 16:39:54 -05:00
Ravi Khadiwala
4d8fe0b6b2 Fix a flaky test 2024-05-07 13:52:35 -05:00
Max Moiseev
411087ff1a Add a testcase with real libsignal User Agent 2024-05-07 14:51:01 -04:00
ravi-signal
10bb2a6a10 Add finer grain rollouts to experiments 2024-05-06 13:28:32 -05:00
Ravi Khadiwala
7aff81547a Manage some unmanaged executors 2024-05-06 13:25:18 -05:00
Ravi Khadiwala
fc097db2a0 Use storage-manager's copy implementation 2024-05-06 13:15:42 -05:00
Alan Liu
843151859d Adding hostname property to cloudflare turn config 2024-05-02 12:35:32 -07:00
adel-signal
854ab353b3 calling: update TurnCallRouter to shuffle instance IPs to prevent allocation skew
Co-authored-by: Jonathan Klabunde Tomer <125505367+jkt-signal@users.noreply.github.com>
2024-05-02 12:34:34 -07:00
Ameya Lokare
cc6ec1d351 Update to the latest version of the spam filter 2024-05-02 12:24:54 -07:00
Chris Eager
cf307db31d Update JsonMappingExceptionMapper “Early EOF” logic 2024-05-01 10:31:11 -05:00
Chris Eager
dcfca4d95e Update to the latest version of the spam filter 2024-05-01 10:30:53 -05:00
Chris Eager
567c368a81 Add DiscoverableSubtypeResolver to static YAML_MAPPER 2024-05-01 10:21:55 -05:00
Chris Eager
223b2fc263 Add javax.validation.Validator to SpamFilter#configure 2024-05-01 10:21:55 -05:00
Chris Eager
4a28ab6317 Add support to trial Cloudflare TURN beta 2024-05-01 10:15:01 -05:00
Chris Eager
0986ce12e6 Validate integration test Config 2024-05-01 10:11:15 -05:00
Ravi Khadiwala
37aa3b8e49 Default to 0 usedBytes in GET /v1/archives 2024-05-01 10:10:46 -05:00
Chris Eager
d7f14339fe Update to the latest version of the spam filter 2024-04-29 11:18:23 -05:00
Chris Eager
0e4be0c85a Add tests for WhisperServerService#run
Additionally, `LocalWhisperServerService` may be used for integration testing.
2024-04-29 11:05:35 -05:00
Jon Chambers
b6f8bca361 Update to the latest version of the spam filter 2024-04-26 15:59:32 -04:00
Jon Chambers
354c72968e Update to the latest version of the spam filter 2024-04-26 15:37:04 -04:00
Jon Chambers
9d3e3c7312 Use a common utility for turning Google API futures into CompletableFutures 2024-04-26 15:27:59 -04:00
Ravi Khadiwala
88e2687e23 Add a check for missing uak in OptionalAccess 2024-04-26 15:24:54 -04:00
Ravi Khadiwala
19944bfdb2 Update to libsignal 0.45 and use libsignal's BackupLevel 2024-04-25 16:54:41 -05:00
Ravi Khadiwala
c8efcf5105 Don't map a Mono<Void> 2024-04-25 16:47:34 -05:00
Jon Chambers
7f6da52349 Publish donation events for immediately-successful donations 2024-04-24 13:16:37 -04:00
Jon Chambers
8999f0104f Trigger pub/sub events for one-time donations via Braintree (PayPal) 2024-04-24 09:19:21 -04:00
Jon Chambers
516c481e94 Pass a CurrencyConversionManager to BraintreeManager 2024-04-24 09:19:21 -04:00
Jon Chambers
3266c2cd8f Add cloud pubsub as a dependency 2024-04-24 09:19:21 -04:00
Jonathan Klabunde Tomer
f0dcd8e07b Group Send Endorsement support for unversioned profile fetch 2024-04-23 14:58:19 -07:00
ravi-signal
9ef1fee172 Add DELETE v1/archives 2024-04-23 16:50:11 -05:00
Jonathan Klabunde Tomer
b3bd4ccc17 simplify profile auth 2024-04-23 14:49:04 -07:00
Chris Eager
fba7686390 Remove shard tag from Lettuce circuit breaker metrics 2024-04-23 16:00:46 -05:00
Ameya Lokare
2d314e5309 Minor: Return exception instead of throwing it in .orElseThrow()
Gets rid of an IntelliJ warning about it. No difference in behavior, there is
a test that already covers this path.
2024-04-22 09:32:12 -04:00
Jon Chambers
ed72d7f9ec Attach client platforms when creating donations 2024-04-22 09:31:57 -04:00
Jonathan Klabunde Tomer
b8f64fe3d4 Group Send Endorsement support for pre-key fetch endpoint 2024-04-19 15:40:46 -07:00
Chris Eager
ab64828661 Update custom Gauges to Micrometer MeterBinders 2024-04-17 17:43:34 -05:00
Chris Eager
10dfa18e81 Update to the latest version of the spam filter 2024-04-17 15:53:12 -05:00
Chris Eager
a38bf25e68 Migrate remaining custom metrics from Dropwizard to Micrometer
And remove some that are obsolete or duplicative.
2024-04-17 15:35:04 -05:00
Ameya Lokare
419ec6e308 Update to the latest version of the spam filter 2024-04-17 10:14:45 -07:00
Jonathan Klabunde Tomer
ada589d0c3 accept Group Send Endorsements for single-recipient message send
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2024-04-16 15:06:40 -07:00
Chris Eager
7068d27a8b Update to the latest version of the spam filter 2024-04-16 12:30:14 -05:00
Chris Eager
a302275187 Use a single cluster instance in MessagesCache 2024-04-16 12:04:18 -05:00
Chris Eager
b734d58ab7 Coalesce all Redis clusters to per-shard circuit breakers 2024-04-16 12:04:18 -05:00
Ravi Khadiwala
2046b02bd8 Rename RedeemReceiptRequest to help openapi 2024-04-16 11:16:18 -05:00
Chris Eager
1df824db7c Remap some JsonMappingExceptions to 408 and 400 2024-04-15 16:01:09 -05:00
Chris Eager
9cad2c6b7d Improve test Redis cluster setup and teardown 2024-04-15 15:58:23 -05:00
Chris Eager
82881c030a Update to the latest version of the spam filter 2024-04-15 14:13:09 -05:00
ravi-signal
00ca7d5942 Add cdn number query parameter to /archives/auth/read 2024-04-15 13:59:14 -05:00
ravi-signal
d36df3eaa9 Add new upload-for-copy backup endpoint 2024-04-15 13:47:46 -05:00
ravi-signal
e5d654f0c7 Add /v1/archives/redeem-receipt 2024-04-15 13:47:02 -05:00
Chris Eager
fc1f471369 Use per-shard circuit breakers for additional Redis clusters 2024-04-15 13:45:24 -05:00
Ameya Lokare
be6f4e38b8 Update to the latest version of the spam filter 2024-04-12 11:29:18 -07:00
Chris Eager
faa8674f39 Update to the latest version of the spam filter 2024-04-12 11:34:39 -05:00
Chris Eager
2dc707d86e Add per-shard Redis circuit breakers 2024-04-12 11:22:41 -05:00
Chris Eager
05a92494bb Remove X-Forwarded-For from RemoteAddressFilter 2024-04-11 11:03:37 -05:00
Alex Konradi
39fd955f13 Allow clients to request zkc-based auth creds
Allow clients to pass a zkcCredential=true query parameter to request the new
auth credential format implemented with the zkcredential Rust library.
2024-04-11 11:00:10 -05:00
ravi-signal
4863e1d227 Add backup levels to subscription configuration response 2024-04-11 10:58:40 -05:00
Jonathan Klabunde Tomer
44ad9d4f5f Update to the latest version of the spam filter 2024-04-10 16:52:28 -07:00
Jonathan Klabunde Tomer
2b652fe2a9 accept group send endorsements for multi-recipient sends 2024-04-10 16:51:09 -07:00
Ravi Khadiwala
cdd2082b07 Decrease logging level in RemoveExpiredBackupsCommand 2024-04-10 18:41:29 -05:00
Chris Eager
5c74aed8f6 Update to the latest version of the spam filter 2024-04-04 16:37:27 -05:00
Chris Eager
5b97bc04e0 Add ExternalRequestFilter 2024-04-04 16:24:20 -05:00
Ravi Khadiwala
63c8b275d1 Return 401 instead of 404 on unknown backup-ids 2024-04-04 10:56:48 -05:00
Ravi Khadiwala
1ebc17352f Check presentation before verifying the signature 2024-04-04 10:56:48 -05:00
Ravi Khadiwala
268c8382ee Fix Backup expiration purge time 2024-04-04 10:55:55 -05:00
ravi-signal
498dcbbfe8 Make media encrypter stream ordered
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2024-04-04 10:53:45 -05:00
Ravi Khadiwala
3a1ecb342f allow striping clients in FaultTolerantHttpClient 2024-04-04 10:47:34 -05:00
ravi-signal
bb0da69c9e Set the shutdown gauge earlier in the shutdown process
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2024-04-02 09:39:55 -05:00
Jon Chambers
796dce3cd3 Always use the "peek" strategy for counting one-time pre-keys 2024-04-02 10:31:20 -04:00
Jon Chambers
f59c34004d De-idiom-ize a "wait for everything to finish" idiom 2024-04-02 10:30:44 -04:00
Ameya Lokare
c4cbf0d618 Update to the latest version of the spam filter 2024-04-01 15:09:44 -07:00
Katherine
d002e5dda8 Hardcode paymentActivation flag to true 2024-04-01 15:07:11 -07:00
Chris Eager
a9d0ab271d Update embedded-redis to 0.9.0 2024-04-01 17:06:55 -05:00
Chris Eager
89cb821c97 Remove vavr dependency 2024-04-01 17:06:55 -05:00
Chris Eager
ef8c520b59 Update reactor-bom to 2023.0.4 2024-04-01 17:06:55 -05:00
Chris Eager
8897fd75ad Update push to 0.15.4 2024-04-01 17:06:55 -05:00
Chris Eager
fd748c1dc3 Update netty to 4.1.108.Final 2024-04-01 17:06:55 -05:00
Chris Eager
c95dbf7508 Update resilience4j to 2.2.0 2024-04-01 17:06:55 -05:00
Chris Eager
ed64c38950 Update Micrometer to 1.12.4 2024-04-01 17:06:55 -05:00
Chris Eager
0b5be8cdcd Update Lettuce to 6.3.2.RELEASE 2024-04-01 17:06:55 -05:00
Chris Eager
fcc77052a6 Add org.eclipse.jetty.io.EofException to expected measureSendMessageErrors 2024-04-01 17:06:29 -05:00
Ravi Khadiwala
831c9ff5bf Make backupDir/mediaDir indirect 2024-04-01 13:45:21 -05:00
Ravi Khadiwala
de37141812 Add a crawler that expires old backups 2024-04-01 13:45:21 -05:00
Ameya Lokare
c35a648734 Update to the latest version of the spam filter 2024-03-27 15:44:31 -07:00
Ravi Khadiwala
a550caf63f Make sure we close the HTTP/2 stream after cdn read errors 2024-03-27 17:00:37 -05:00
Ravi Khadiwala
de9eaa98db 404 instead of 400 on unknown source cdn 2024-03-27 17:00:37 -05:00
ravi-signal
37b657cbbd avoid baos::writeTo on virtual threads 2024-03-27 16:58:38 -05:00
Jon Chambers
a733f5c615 Add debugging context to signature validation failures 2024-03-27 17:58:02 -04:00
Jon Chambers
8a587d1d12 Rename NoiseStreamHandler to NoiseTransportHandler for consistency with Noise specification terminology 2024-03-27 17:57:46 -04:00
Chris Eager
75bb22f08b Include HTTP2ServerConnectionFactory in JettyHttpConfigurationCustomizer 2024-03-27 16:56:19 -05:00
Jon Chambers
d10da39e5b Pare back debug-oriented metrics around fetching pre-keys 2024-03-26 16:44:34 -04:00
adel-signal
54e9b839bd update TurnTokenGenerator to add whether this is with ip or url turn allocation 2024-03-26 13:40:53 -07:00
Jon Chambers
aec6ac019f Introduce a Noise-over-WebSocket client connection manager 2024-03-22 15:20:55 -04:00
Ravi Khadiwala
075a08884b Preserve backupCredentialRequest across rereg 2024-03-22 14:19:35 -05:00
Jon Chambers
6fcb2ab5dd Remove username phased rollout plumbing (usernames are now available to everybody) 2024-03-21 13:42:20 -04:00
ameya-signal
7f0f045f29 Minor cleanup of report spam endpoint (#1979) 2024-03-21 10:41:25 -07:00
Ravi Khadiwala
e7d1eadf8e Fix experiments in BackupAuthManager 2024-03-20 11:43:07 -05:00
Chris Eager
a9b5359f7c Update to the latest version of the spam filter 2024-03-19 13:32:33 -05:00
Chris Eager
9df6e19204 Ignored EofException response failure in MetricsHttpChannelListener 2024-03-18 17:38:19 -05:00
Chris Eager
5eaae184c9 Set request failure to debug in MetricsHttpChannelListener 2024-03-18 17:38:19 -05:00
Chris Eager
459882e6fa Add JettyHttpConfigurationCustomizer 2024-03-18 17:38:19 -05:00
Chris Eager
2c2b5d555e Rename obsolete uses of recaptcha 2024-03-13 16:40:32 -05:00
Alex Konradi
0ab2428d87 Don't produce zkgroup auth creds with PNI as ACI 2024-03-13 16:22:45 -05:00
Chris Eager
8574494573 Support "captcha" rename in AnswerChallengeRequest.type 2024-03-13 16:19:05 -05:00
Chris Eager
a4d4a9c686 Remove RecaptchaClient 2024-03-13 16:19:05 -05:00
Chris Eager
3d32b68bb2 Move WebSocketClient SecureRandom to a static field 2024-03-12 16:54:38 -05:00
Chris Eager
fd9eb462cc Replace extraneous SecureRandom with empty byte array 2024-03-12 16:49:57 -05:00
Ravi Khadiwala
f9533e016f Adjust metrics in RemoveExpiredUsernameHoldsCommand 2024-03-11 13:50:08 -05:00
Jon Chambers
85b15fa63b Actually increment the "get group credentials" counter 2024-03-08 17:07:16 -05:00
Chris Eager
e236842888 Update to the latest version of the spam filter 2024-03-08 14:11:56 -06:00
Chris Eager
3dadaf9334 Migrate DynamicConfigurationManager to use java.util.concurrent 2024-03-08 13:53:28 -06:00
Ravi Khadiwala
9e510a678c disable response buffering on the websocket
Jersey buffers responses (by default up to 8192 bytes) just so it can
add a content length to responses. We already buffer our responses to
serialize them as protos, so we can compute the content length
ourselves. Setting the buffer to zero disables buffering.
2024-03-08 13:46:00 -06:00
Chris Eager
2dc0ea2b89 Address potential NullPointerException when calling Collection#contains 2024-03-08 13:43:31 -06:00
Alex Konradi
7d364ca7ce Count group auth reqs without pniAsServiceId=true 2024-03-06 15:41:48 -05:00
Chris Eager
9f6a6d7f5b Include HTTP/2 stream idle timeouts in IOExceptionMapper 2024-03-06 11:11:39 -06:00
Ravi Khadiwala
3cc740cda3 Temporarily hold a username after an account releases it 2024-03-06 11:09:25 -06:00
Ravi Khadiwala
47b24b5dff Simplify username operations in Accounts
- Group username table constants together
- Rethrow JsonProcessingException earlier
- Use UpdateAccountSpec.forAccount in username operations
- Inline confirm/clear transaction helpers
2024-03-06 11:09:25 -06:00
adel-signal
8f100a792e calling: update TurnCallRouter to reduce returned options 2024-03-05 11:26:19 -08:00
Chris Eager
84c6731ddf Update protobuf.version to match upstream 2024-02-29 16:56:25 -06:00
Chris Eager
1f1de353de Reapply "Update to Dropwizard 3.0.6"
This reverts commit 95e83c52fa.
2024-02-29 16:56:25 -06:00
Chris Eager
40eb82adbe Update actions/checkout and actions/setup-java to 4.x 2024-02-29 15:56:55 -06:00
Ehren Kret
d9240e1e2e use consistent abbreviation for GNU AGPLv3 2024-02-28 18:09:27 -06:00
Ehren Kret
9abaed8385 update README copyright year 2024-02-28 17:51:05 -06:00
Ravi Khadiwala
95e83c52fa Revert "Update to Dropwizard 3.0.6"
This reverts commit 2ad5d33251.
2024-02-28 15:58:55 -06:00
Ravi Khadiwala
be377dcda8 Update to the latest version of the spam filter 2024-02-28 15:34:32 -06:00
Chris Eager
88a68e883e Update various <exclusions> in POM 2024-02-28 15:25:33 -06:00
Chris Eager
2ad5d33251 Update to Dropwizard 3.0.6 2024-02-28 15:25:33 -06:00
Ravi Khadiwala
4e5dd914dd Add varargs variant to HmacUtil 2024-02-28 15:18:39 -06:00
ravi-signal
2adf1e5017 Avoid modification of Account from @ReadOnly endpoint 2024-02-28 15:18:12 -06:00
Jon Chambers
55ca4e93c4 Update to noise-java 0.1.1 2024-02-28 16:16:46 -05:00
Jon Chambers
d1d03f45c5 Resolve warnings throughout AccountControllerTest 2024-02-26 16:11:03 -05:00
Ravi Khadiwala
436bd891bd Don't register SpamFilter as a request filter 2024-02-26 09:33:36 -06:00
Chris Eager
a7c28fe5ed Rename "name" tags to be more distinguishing 2024-02-26 09:32:50 -06:00
Chris Eager
60814d1ff0 Configure metrics registries earlier in background command setup 2024-02-26 09:32:01 -06:00
Jon Chambers
d018efe2a5 Require non-null proofs in "confirm username hash" requests 2024-02-26 10:30:52 -05:00
Jon Chambers
6fd0cba06a Temporarily restore the giftBadges capability for backward compatibility 2024-02-26 10:30:29 -05:00
Ravi Khadiwala
86f9322036 Update to the latest version of the spam filter 2024-02-23 16:08:41 -06:00
Katherine
12c6af23ee Map TransactionConflict to ContestedOptimisticLockException in username flows 2024-02-23 14:06:03 -08:00
Ravi Khadiwala
69330f47fd Explicitly call spam-filter for verification session updates
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-23 16:04:24 -06:00
Ravi Khadiwala
4f40c128bf Explicitly call spam-filter for challenges
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-23 15:58:52 -06:00
Jon Chambers
30b5ad1515 Fix an incorrectly-named "check keys" method 2024-02-23 13:17:10 -05:00
adel-signal
665a26d164 update call routing to return ipv6 ips in RFC3986 format 2024-02-23 11:57:58 -05:00
Jon Chambers
a5774bf6ff Introduce a (dormant) Noise/WebSocket for future client/server communication 2024-02-23 11:42:42 -05:00
Jon Chambers
d2716fe5cf Add an endpoint for checking that clients and the server have a common view of the client's repeated-use keys 2024-02-23 11:41:58 -05:00
Jon Chambers
279f877bf2 Validate pre-key signatures via the legacy "set signed pre-key" endpoint 2024-02-23 11:33:41 -05:00
Ravi Khadiwala
d51e6a43e7 Update to the latest version of the spam filter 2024-02-22 12:12:42 -06:00
Ravi Khadiwala
6a96756c87 Update to the latest version of the spam filter 2024-02-22 12:09:37 -06:00
Ravi Khadiwala
df69d9f195 Annotate authenticated endpoints with @ReadOnly or @Mutable 2024-02-22 12:05:32 -06:00
Ravi Khadiwala
26ffa19f36 Lifecycle management for Account objects reused accross websocket requests 2024-02-22 12:05:32 -06:00
Jon Chambers
29ef3f0b41 Add tests and metrics for parsing invalid keys 2024-02-21 15:32:25 -05:00
Jon Chambers
106d5e54c7 Extract a common base class for public key serializers/deserializers 2024-02-21 15:32:25 -05:00
Jon Chambers
6ac2460eb0 Drop the "key without version byte" counter 2024-02-21 15:32:25 -05:00
dependabot[bot]
79c030b138 Bump org.apache.commons:commons-compress from 1.24.0 to 1.26.0 in /service 2024-02-21 11:52:41 -08:00
Chris Eager
c8d649e8c2 Update MetricsHttpChannelListener to share MetricsRequestEventListener namespace 2024-02-21 13:32:47 -06:00
Ravi Khadiwala
1fdf82dd6c Remove unused ChangePhoneNumberRequest object
Actual request object is ChangeNumberRequest
2024-02-21 13:32:27 -06:00
Jon Chambers
4aa4246695 Clear account records from the account cache after username operations 2024-02-20 12:11:29 -05:00
Chris Eager
1bebceb29c Actually fix flaky test in MetricsHttpChannelListenerIntegrationTest 2024-02-20 10:52:04 -06:00
Jon Chambers
a2139ee236 Revert "Add diagnostic dimensions to the "get keys" counter"
This reverts commit cd64390141.
2024-02-18 20:14:05 -05:00
Jon Chambers
8c55f39cdf Revert "Use a phased enrollment strategy for the pnp compatibility flag"
This reverts commit 3e12a8780d.
2024-02-18 20:08:39 -05:00
Chris Eager
0329184c94 Fix flaky test in MetricsHttpChannelListenerIntegrationTest 2024-02-18 20:08:29 -05:00
Jon Chambers
cd64390141 Add diagnostic dimensions to the "get keys" counter 2024-02-18 18:10:58 -05:00
Jon Chambers
3e12a8780d Use a phased enrollment strategy for the pnp compatibility flag 2024-02-18 17:59:00 -05:00
Jon Chambers
11e6ff1bbe Introduce a pnp capability for backward compatibility 2024-02-18 17:59:00 -05:00
Jon Chambers
36f85fc97e Fix an inverted boolean in a counter 2024-02-16 15:18:18 -05:00
Jon Chambers
9040cfd200 Count "store key" attempts by key type/platform 2024-02-16 13:09:55 -05:00
Jon Chambers
757da3b15a Fully-qualify OpenAPI Tag, allowing for imported Micrometer Tag 2024-02-16 13:09:55 -05:00
Jon Chambers
d162590a32 Record a remote-aggregated distribution of one-time pre-key counts 2024-02-16 13:09:55 -05:00
Jon Chambers
f41e1716c6 Combine/expand existing "get keys" counter 2024-02-16 13:09:55 -05:00
Jon Chambers
4dce0f1b9d Add identity type/"stale" dimensions to the "pre-key unavailable" counter 2024-02-16 13:09:55 -05:00
Jonathan Klabunde Tomer
fef57dce0d use explicit Timer rather than micrometer annotation for send-message latency distribution 2024-02-15 14:58:43 -08:00
Jon Chambers
d884700b61 Significantly reduce default Redis command timeout 2024-02-15 17:57:17 -05:00
Jonathan Klabunde Tomer
ff9ad4bd1d export 1:1 message send latency as a full distribution 2024-02-15 10:33:02 -08:00
Chris Eager
9ce2b7555c Add static servlet paths to MetricsHttpChannelListener 2024-02-14 17:09:30 -06:00
Jon Chambers
f90ccd3391 Count attempts to fetch a one-time EC pre-key that result in a "key unavailable" response 2024-02-14 18:04:31 -05:00
Jon Chambers
5ff092e541 Retire the pni device capability 2024-02-14 18:04:23 -05:00
Ravi Khadiwala
dcdf401f64 Make DELETE /v1/device synchronous 2024-02-14 15:30:10 -05:00
Chris Eager
e4fb80b39b Add onResponseFailure handler to MetricsHttpChannelListener 2024-02-14 13:43:23 -06:00
Chris Eager
9745854ab8 Switch MetricsHttpChannelListener to ContainerResponseFilter 2024-02-14 13:43:23 -06:00
Jon Chambers
7124621f66 Use pre-calculated pre-key counts when possible 2024-02-14 14:28:33 -05:00
Jon Chambers
47fd8f5793 Assume that all devices are PNI-capable 2024-02-14 14:27:48 -05:00
Jonathan Klabunde Tomer
40d698f2db Remove last vestiges of stale capabilities.
Not for deployment until 2024-02-06!
2024-02-14 14:23:05 -05:00
adel-signal
74abe98706 initialize call routing data syncs 2024-02-13 17:05:08 -08:00
Chris Eager
86787f3bc8 Start DynamicConfigurationManager directly after construction 2024-02-13 13:08:20 -06:00
Chris Eager
699b0c775a Add dynamic configuration to enable detailed Lettuce metrics 2024-02-13 10:33:43 -06:00
Chris Eager
ff59ef8094 Add MetricHttpChannelListener 2024-02-13 10:29:03 -06:00
Jonathan Klabunde Tomer
089af7cc1f correct metric name in MaxMindDatabaseManager timer 2024-02-09 13:37:39 -08:00
Jonathan Klabunde Tomer
1591a2d9a3 CallRoutingTableManager: fix crash on startup due to typo in metric naming 2024-02-09 12:27:51 -08:00
Jonathan Klabunde Tomer
f7984ed642 CallDnsRecordsManager: fix crash on startup due to typo in metric naming 2024-02-09 12:14:50 -08:00
Jonathan Klabunde Tomer
be634c6043 Update to the latest version of the spam filter 2024-02-09 11:38:18 -08:00
adel-signal
d1f68eacd9 Add call routing API endpoint for turn servers 2024-02-09 11:12:23 -08:00
Jon Chambers
4f45f23094 Prevent old versions of the Android app from carrying out "change phone number" requests 2024-02-09 09:58:38 -05:00
Jon Chambers
c5dc01ee11 Restore high-cardinality Lettuce metrics for debugging 2024-02-08 19:27:56 -05:00
Ravi Khadiwala
587c385936 Update to the latest version of the spam filter 2024-02-07 17:19:59 -06:00
Ravi Khadiwala
3a641a58b0 Update to the latest version of the spam filter 2024-02-07 14:38:35 -06:00
Chris Eager
e944306a28 Remove obsolete dropwizard-db dependency 2024-02-07 14:35:23 -06:00
Ravi Khadiwala
3b44ed6d16 Explicitly call spam-filter for messages
Pass in the same information to the spam-filter, but just use explicit
method calls rather than jersey request filters.
2024-02-07 14:30:32 -06:00
Chris Eager
0965ab8063 Move HeaderUtils#getMostRecentyProxy to RemoteAddressFilter 2024-02-07 14:28:37 -06:00
Chris Eager
fcae100df1 Add dropwizard-http2 dependency 2024-02-07 14:28:20 -06:00
Fedor Indutny
24a7762873 Add identityType tag to SENT_MESSAGE_COUNTER_NAME 2024-02-06 17:39:00 -06:00
Chris Eager
e441ab60a2 Add metric for IPv4/IPv6 requests count 2024-02-06 17:38:25 -06:00
Chris Eager
50c2bc5edb Update to the latest version of the spam filter 2024-02-05 13:45:50 -06:00
Chris Eager
2ab14ca59e Refactor remote address/X-Forwarded-For handling 2024-02-05 13:37:06 -06:00
Chris Eager
4475d65780 Make Accounts#changeNumber exception handling more resilient to future changes 2024-02-01 15:14:15 -06:00
Ravi Khadiwala
b1d10f5817 Add lifecycle management for AwsCrt for commands 2024-02-01 15:08:45 -06:00
Chris Eager
36664f37de Update to the latest version of the spam filter 2024-01-31 18:04:03 -06:00
Chris Eager
c838df90ef Add HttpServletRequestUtil 2024-01-31 17:53:47 -06:00
Chris Eager
fb39af67e5 Allow for more TransactWriteItems when inspecting TransactionCanceledException 2024-01-31 17:46:32 -06:00
Chris Eager
2d4d37f96a Translate TransactionCanceledException to ContestedOptimisticLockException in Accounts#changeNumber 2024-01-31 17:27:16 -06:00
Jonathan Klabunde Tomer
84af984c4b remove unversioned and libsignal-0.36 versioned zk secrets from sample bundle 2024-01-31 15:25:23 -08:00
Jon Chambers
26adf20ee8 Make "fetch profile" endpoints @ManagedAsync 2024-01-31 14:38:50 -06:00
Jon Chambers
72668ed0a2 Make "send individual message" endpoint @ManagedAsync 2024-01-31 14:38:50 -06:00
Ravi Khadiwala
50f1ed7851 Add a virtual-thread backed @ManagedAsyncExecutor 2024-01-31 14:38:50 -06:00
ravi-signal
cf8f2a3463 remove synchronized locks that may be held while blocking 2024-01-31 14:29:15 -06:00
ravi-signal
b483159b3a reintroduce VirtualThreadPinEventMonitor 2024-01-31 14:28:12 -06:00
ravi-signal
480abebf7e Offload account lock updates to accountLockExecutor 2024-01-31 14:27:16 -06:00
Ravi Khadiwala
b924dea045 Remove VirtualThreadPinEventMonitor 2024-01-30 13:49:39 -06:00
Ravi Khadiwala
2c1e7e5ed6 Disable flaky VirtualThreadPinEventMonitorTest 2024-01-30 13:29:58 -06:00
Ravi Khadiwala
4dfd74906c Add timeouts to VirtualThreadPinEventMonitorTest 2024-01-30 13:10:43 -06:00
Chris Eager
fdae6ad94f Update to the latest version of the spam filter 2024-01-30 12:53:01 -06:00
Chris Eager
c80225a18c Remove NoneHealthResponder 2024-01-30 12:51:43 -06:00
ravi-signal
0e6242373e Add a monitor for virtual thread pin events 2024-01-30 12:48:07 -06:00
ravi-signal
4305db5579 Update to aws sdk 2.23.8 and use AwsCrtHttpClient 2024-01-30 12:46:27 -06:00
ravi-signal
36e7772f74 Put some validation on to profile version strings
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2024-01-26 15:24:38 -06:00
Chris Eager
ca05df5172 Revert "Add support for secondary credentials for registration service"
This reverts commit 4b8fc2950f.
2024-01-26 11:42:35 -06:00
Chris Eager
422e8e6f3e Remove CommandStopListener 2024-01-26 11:31:04 -06:00
Jon Chambers
852b285d84 Configure and instantiate a ClientPublicKeys data store/manager 2024-01-26 10:29:59 -05:00
Jon Chambers
6c13193623 Add a data store for client public keys for transport-level authentication/encryption 2024-01-26 10:29:59 -05:00
Ravi Khadiwala
61809107c8 Update to the latest version of the spam filter 2024-01-25 13:27:13 -06:00
Ravi Khadiwala
6bda9d8604 Set destination account on the request context 2024-01-25 13:02:28 -06:00
Ravi Khadiwala
1428ca73de Make identity token fetcher more async friendly
After the identity token expires a subsequent call would do a blocking
operation to retrieve the new token. Since we're making use of an async
gRPC client, this tends to block a thread we don't want to be blocking
on.

Instead, switch to periodically refreshing the token on a dedicated
thread.
2024-01-25 13:01:21 -06:00
Katherine
498ace0488 Remove iOS special case handling logic for SEPA/iDEAL 2024-01-25 10:52:17 -08:00
Chris Eager
f082b95efb Update to the latest version of the spam filter 2024-01-24 15:01:35 -06:00
Chris Eager
4b8fc2950f Add support for secondary credentials for registration service 2024-01-24 13:52:23 -06:00
Jon Chambers
595cc55578 Retire the returnPqKey flag when fetching pre-keys 2024-01-23 16:01:40 -05:00
Ravi Khadiwala
91b0c368b4 Remove unused table configuration parameter 2024-01-23 14:48:52 -06:00
Jonathan Klabunde Tomer
21d0ffc990 Allow "reserving" one's own committed username hash 2024-01-23 12:44:55 -08:00
Chris Eager
55b9d84956 Add host to DogstatsdConfiguration 2024-01-22 15:59:52 -06:00
Chris Eager
ffdb0db6c6 Migrate from host tag to dd.internal.entity_id 2024-01-22 15:59:52 -06:00
Chris Eager
a5ed07a666 Support environment variable substitution in configuration 2024-01-22 11:25:18 -06:00
Chris Eager
da02c90bad Remove AWS instance profile credentials provider 2024-01-22 11:24:03 -06:00
Ravi Khadiwala
3820a231ec Strictly enforce ACI service identifier strings have no prefix 2024-01-22 11:23:33 -06:00
Dimitris Apostolou
408b065b9e Fix typos 2024-01-20 12:40:08 -05:00
hduelme
238ab84749 use StandardCharsets UTF_8 instead of String 2024-01-20 12:39:43 -05:00
Ravi Khadiwala
6894015986 Update to the latest version of the spam filter 2024-01-19 14:32:15 -06:00
Ravi Khadiwala
f5080f9bd6 Support configurable verification code sender overrides 2024-01-19 13:58:17 -06:00
Ravi Khadiwala
db4aa99ce0 Make the ServiceIdentifier interface sealed 2024-01-19 13:57:20 -06:00
ravi-signal
70134507f8 Add metric for ServiceId string parsing 2024-01-19 13:56:47 -06:00
Ravi Khadiwala
360a4793ae Remove global lock in RateLimitChallengeManager 2024-01-19 13:56:09 -06:00
Jon Chambers
47bfb25f2c Retire a now-unused transacational update method for accounts 2024-01-18 16:40:21 -05:00
Jon Chambers
b048b0bf65 Remove signed pre-keys from Device entities 2024-01-18 12:13:00 -05:00
Jonathan Klabunde Tomer
394f9929ad limit size of multi-recipient messages 2024-01-11 16:31:37 -08:00
ravi-signal
bf39be3320 Add Content-Type header for copy uploads 2024-01-11 14:59:35 -06:00
Chris Eager
4a2cbb9ec7 Stop timers on Flux termination 2024-01-11 14:57:31 -06:00
Ravi Khadiwala
cc6cf8194f Add media deletion endpoint 2024-01-11 10:19:06 -06:00
Ravi Khadiwala
e934ead85c Fix incorrect comparison in archive usage metric 2024-01-11 10:19:06 -06:00
Ravi Khadiwala
323bfd9a6e Remove some secondary keys from account cache
Remove e164, usernameHash, and usernameLink secondary mappings from the
accounts redis cache.
2024-01-11 10:10:32 -06:00
Jon Chambers
bf05e47e26 Get accounts asynchronously when checking SVR credentials via gRPC 2024-01-11 11:09:49 -05:00
Jon Chambers
d18f576239 Revert "Revert "Treat the stand-alone signed pre-keys table as the source of truth for signed pre-keys""
This reverts commit 3f9edfe597.
2024-01-10 11:22:10 -05:00
Jon Chambers
7d483c711a Revert "Revert "Retire "migrate signed pre-keys" configuration""
This reverts commit a024949311.
2024-01-10 11:22:10 -05:00
Jon Chambers
61256d49cd Revert "Revert "Retire the "migrate signed pre-keys" command""
This reverts commit f738bc97e7.
2024-01-10 11:22:10 -05:00
Jonathan Klabunde Tomer
184cdc0331 Remove two-stage check of username availability in reserve/confirm 2024-01-09 14:01:42 -08:00
Ravi Khadiwala
ed972a0037 Fix archive listing directory prefix order 2024-01-09 16:01:11 -06:00
Ravi Khadiwala
a62a6c1cb6 Change type of Cdn3StorageManagerConfiguration.clientSecret to SecretString 2024-01-08 16:06:54 -06:00
Ravi Khadiwala
ba0c6be3e3 Update to the latest version of the spam filter 2024-01-08 15:00:29 -06:00
Ravi Khadiwala
f66566aa17 Fix default configuratiton in MonitoredS3ObjectConfiguration 2024-01-08 14:52:14 -06:00
ravi-signal
b6ecfc7131 Add archive listing 2024-01-08 13:54:57 -06:00
adel-signal
460dc6224c Update to the latest version of the spam filter 2024-01-08 10:56:43 -08:00
adel-signal
2b688b1a60 Refactor spam filter's S3MonitoredObject to server parent module.
Allows other parts of server to use S3MonitoredObjects.
2024-01-08 10:34:12 -08:00
Jon Chambers
3c64d9292f Revert "Expand the default max packet size for Dogstatsd"
This reverts commit 0f52d2e464.
2024-01-08 12:23:16 -05:00
Jon Chambers
0f52d2e464 Expand the default max packet size for Dogstatsd 2024-01-08 11:43:50 -05:00
Jonathan Klabunde Tomer
1e5fadc440 Allow reserving a hash previously reserved (but not committed) by the same user 2024-01-05 15:38:45 -08:00
Chris Eager
f495ff483a Update RemoveExpiredLinkedDevicesCommand to retry failures 2024-01-05 15:38:34 -08:00
Jonathan Klabunde Tomer
4e3b1509a8 Update to the latest version of the spam filter 2024-01-04 12:20:50 -08:00
Jonathan Klabunde Tomer
d1a80cc880 fix build-breaking typo 2024-01-04 11:46:07 -08:00
Jonathan Klabunde Tomer
e1ad25cee0 Group Send Credential support in chat 2024-01-04 11:38:57 -08:00
Chris Eager
195f23c347 Add /v1/accounts and /v2/keys to the WebSocket 2024-01-02 15:51:57 -06:00
Chris Eager
ad6b99be6a Dispatch client presence operations after device deletion to a dedicated executor 2023-12-22 10:51:17 -06:00
Chris Eager
b9dd9fc47d Reduce fan-out by processing a single stream of expired linked devices 2023-12-22 10:51:06 -06:00
Ravi Khadiwala
19a8a80a30 Update to the latest version of the spam filter 2023-12-21 16:29:54 -06:00
Chris Eager
637792c6d4 Move RemoveExpiredLinkedDevicesCommand error handling for more accurate metrics 2023-12-21 16:06:41 -06:00
Jon Chambers
4d1bca2d97 Dispatch client presence operations after account deletion to a dedicated executor 2023-12-21 13:40:49 -05:00
Chris Eager
f33a2eba50 Add buffer + shuffle and configurable concurrency to RemoveExpiredLinkedDevicesCommand 2023-12-21 11:14:02 -06:00
Jon Chambers
5d6bea5ec9 Clear Redis cache entries strictly after removing accounts from DynamoDB 2023-12-21 12:08:03 -05:00
Jon Chambers
057d1f07a8 Remove bulk "set repeated-use signed pre-keys" methods because they were only ever used for single devices 2023-12-21 12:07:42 -05:00
Ravi Khadiwala
25c3f55672 Update documentation/integration-tests actions to java 21 2023-12-20 16:41:16 -06:00
Ravi Khadiwala
c9d4091c1e Update to the latest version of the spam filter 2023-12-20 16:38:20 -06:00
Ravi Khadiwala
1d55562dc3 Update to Java 21 2023-12-20 16:37:14 -06:00
Jon Chambers
95bb9a9780 Log signed pre-key retrieval errors and mismatches 2023-12-20 14:55:12 -05:00
Chris Eager
06c391cbf6 Add counter for failed account updates to RemoveExpiredLinkedDevicesCommand 2023-12-20 13:51:28 -06:00
Chris Eager
d90dff95b1 Move MeterRegistry#close calls to after the lifecycle has fully stopped 2023-12-20 13:51:16 -06:00
Ravi Khadiwala
c93972a322 Update to the latest version of the spam filter 2023-12-19 18:33:45 -06:00
Ravi Khadiwala
ca47a7b663 handle new RegistrationService proto error 2023-12-19 18:19:26 -06:00
Jonathan Klabunde Tomer
9d3d4a3698 Add phone-number-sharing field to versioned profile
Co-authored-by: Katherine <katherine@signal.org>
2023-12-19 11:20:04 -08:00
Chris Eager
3b509bf820 Add command to remove expired linked devices 2023-12-19 13:11:26 -06:00
Jon Chambers
5b7f91827a Remove signed pre-keys transactionally when removing devices 2023-12-19 14:11:05 -05:00
Chris Eager
a44491714c Update to the latest version of the spam filter 2023-12-18 14:55:39 -06:00
Chris Eager
06800043a9 Set TLS keystore password in secondary persistent services 2023-12-15 13:39:58 -06:00
Chris Eager
3090de56b8 Set TLS keystore password from secrets configuration 2023-12-15 12:47:27 -06:00
Katherine
a37acd1f42 Add ttl for braintree writes to onetime donation table 2023-12-15 13:37:35 -05:00
Jonathan Klabunde Tomer
372e3f83d2 Update libsignal to 0.37.0 2023-12-15 10:36:59 -08:00
Chris Eager
de260a2bef Update to the latest version of the spam filter 2023-12-15 12:23:01 -06:00
Jonathan Klabunde Tomer
e9a130f976 add versioned zkconfig secrets to sample secrets bundle 2023-12-14 17:38:49 -08:00
Jonathan Klabunde Tomer
43f17414ff Make key-setting methods asynchronous again 2023-12-14 16:49:16 -08:00
Jon Chambers
b259eea8ce Refactor/clarify account creation/reclamation process 2023-12-14 16:48:57 -08:00
Katherine
9cfc2ba09a Persist onetime donation payment success timestamps for Braintree transactions 2023-12-14 16:48:29 -08:00
Chris Eager
bb347999ce Propagate another subscription processor error to clients 2023-12-14 15:40:08 -05:00
Katherine
3548c3df15 Calculate onetime badge expiration from payment success timestamp 2023-12-14 15:39:46 -05:00
Jon Chambers
1167d0ac2e Make key-setting methods synchronous 2023-12-13 17:49:55 -05:00
Jon Chambers
f738bc97e7 Revert "Retire the "migrate signed pre-keys" command"
This reverts commit c7cc3002d5.
2023-12-13 17:49:55 -05:00
Jon Chambers
3f9edfe597 Revert "Treat the stand-alone signed pre-keys table as the source of truth for signed pre-keys"
This reverts commit feb933b4df.
2023-12-13 17:49:55 -05:00
Jon Chambers
a024949311 Revert "Retire "migrate signed pre-keys" configuration"
This reverts commit 44145073f1.
2023-12-13 17:49:55 -05:00
Jon Chambers
609c901867 Refactor key-fetching to be reactive 2023-12-13 12:46:48 -05:00
Jon Chambers
4ce060a963 Count wildcard "get keys" requests by platform 2023-12-13 12:46:48 -05:00
Jon Chambers
c4ca0fee40 Synchronize access to responseItems when assembling a "get keys" response 2023-12-13 12:46:48 -05:00
Chris Eager
8d4acf0330 Remove ForkJoinPool.managedBlock in favor of async updates 2023-12-13 10:18:04 -05:00
Jon Chambers
28a981f29f Assume that all devices have signed pre-keys 2023-12-13 10:17:51 -05:00
Jon Chambers
c29113d17a Reject requests with missing device capabilities 2023-12-12 11:37:44 -05:00
Katherine
951f978447 Use start of subscription period as fallback timestamp 2023-12-12 11:34:44 -05:00
Jon Chambers
07899f35bd Return DeviceController#linkDevice to synchronous processing 2023-12-12 11:34:23 -05:00
Katherine
3cbbf37468 Use payment success timestamp to calculate recurring donation badge expiration 2023-12-12 10:01:20 -05:00
Jon Chambers
6c7a3df5ae Retire non-atomic device-linking pathways 2023-12-12 09:53:51 -05:00
Jon Chambers
2054ab2771 Revert "Count requests with missing device capabilities"
This reverts commit 6cdf8ebd2c.
2023-12-12 09:53:51 -05:00
Jon Chambers
44145073f1 Retire "migrate signed pre-keys" configuration 2023-12-12 09:53:34 -05:00
Jon Chambers
feb933b4df Treat the stand-alone signed pre-keys table as the source of truth for signed pre-keys 2023-12-11 11:39:54 -05:00
Jon Chambers
c7cc3002d5 Retire the "migrate signed pre-keys" command 2023-12-11 11:07:00 -05:00
xiaolou86
049b901d63 Fix typos 2023-12-09 10:04:12 -05:00
Jon Chambers
3cf1b92dfc Do not set one-time pre-keys if the lists of new keys are empty 2023-12-08 14:03:08 -05:00
Jon Chambers
5b0fcbe854 Always expose sequential fluxes to account crawlers 2023-12-08 13:43:42 -05:00
Jon Chambers
cca747a1f6 Treat transaction conflicts during transactional account updates as contested optimistic locks 2023-12-08 12:13:23 -05:00
Jon Chambers
417d99a17e Check story rate limits in parallel 2023-12-08 12:13:08 -05:00
Jon Chambers
e9708b9259 Use random UUIDs throughout MessageControllerTest 2023-12-08 12:13:08 -05:00
Jon Chambers
e5d3be16b0 Fetch destination accounts in parallel when sending multi-recipient messages 2023-12-08 12:13:08 -05:00
Jordan Rose
2ab3c97ee8 Replace MultiRecipientMessage parsing with libsignal's implementation
Co-authored-by: Jonathan Klabunde Tomer <jkt@signal.org>
2023-12-08 11:52:47 -05:00
Jon Chambers
f20d3043d6 Process key migrations sequentially to better control concurrency 2023-12-07 21:42:49 -05:00
Jonathan Klabunde Tomer
4efda89358 multisend cleanup 2023-12-07 12:23:02 -08:00
Jon Chambers
1fb88271e5 Invalidate cache entries for accounts after successfully adding devices 2023-12-07 13:27:26 -05:00
Jon Chambers
a843780f68 Add a (failing!) test for device-linking 2023-12-07 13:27:26 -05:00
Jon Chambers
5ad83da4e0 Remove the now-unused admin logger 2023-12-07 12:58:43 -05:00
Jon Chambers
949cc9e214 Update to the latest version of the spam filter 2023-12-07 12:58:23 -05:00
Jon Chambers
50d92265ea Add devices to accounts transactionally 2023-12-07 11:19:40 -05:00
Chris Eager
e084a9f2b6 Remove PUT/DELETE methods from RemoteConfigController 2023-12-07 10:54:19 -05:00
Jon Chambers
664f9f36e1 Use padded base64 encoding for outbound device names 2023-12-07 10:53:56 -05:00
Jon Chambers
4c9efdb936 Revert "Revert "Represent device names as byte arrays""
This reverts commit 45848e7bfe.
2023-12-07 10:53:56 -05:00
Jon Chambers
45848e7bfe Revert "Represent device names as byte arrays"
This reverts commit 5ae2e5281a.
2023-12-06 17:05:04 -05:00
Ravi Khadiwala
4fa10e5783 Fix NPE in request metrics when a finished request is missing a response 2023-12-06 15:11:41 -05:00
Ravi Khadiwala
fc0bc85f4d Add some extra request validation to /v1/archives/keys 2023-12-06 15:11:27 -05:00
Jon Chambers
5ae2e5281a Represent device names as byte arrays 2023-12-06 15:10:45 -05:00
Jon Chambers
34a943832a Align push notification types and delivery priorities 2023-12-06 12:40:16 -05:00
Jon Chambers
db17693ba7 Revert "Temporarily make registration challenge notifications "noisy""
This reverts commit 9069c5abb6.
2023-12-06 12:40:16 -05:00
Jon Chambers
6cdf8ebd2c Count requests with missing device capabilities 2023-12-06 10:40:50 -05:00
Ravi Khadiwala
072b470f46 Be more specific about encodings in /v1/archive docs 2023-12-06 10:40:20 -05:00
Ravi Khadiwala
78b2df2ecc Use long instead of int for epoch seconds 2023-12-06 10:40:06 -05:00
Jon Chambers
51a825f25c Update to the latest version of the spam filter 2023-12-06 10:39:04 -05:00
Jon Chambers
00e72a30c9 Assume all accounts have primary devices 2023-12-06 10:28:19 -05:00
Ravi Khadiwala
69990c23a5 Avoid test dep in integration-test 2023-12-05 16:35:51 -06:00
Jon Chambers
df421e0182 Update signed pre-keys in transactions 2023-12-05 14:20:16 -05:00
Jon Chambers
ede9297139 Disallow identity key changes 2023-12-05 14:14:24 -05:00
Jon Chambers
85383fe581 Remove an unused "store keys" method 2023-12-05 14:14:24 -05:00
Jon Chambers
4cca7aa4bd Normalize identity types throughout KeysController 2023-12-05 14:14:24 -05:00
Jon Chambers
e2037dea6c Rename PreKeyState to SetKeysRequest 2023-12-05 14:14:24 -05:00
Jon Chambers
f10f772e94 Convert PreKeyState to a record 2023-12-05 14:14:24 -05:00
Jon Chambers
9ecfe15ac4 Use multiline strings for PreKeyState documentation 2023-12-05 14:14:24 -05:00
Jon Chambers
5f0726af8a Perform cleanup operations before overwriting an existing account record 2023-12-05 12:18:09 -05:00
Ravi Khadiwala
331bbdd4e6 replace deprecated apache RandomUtils 2023-12-05 10:48:18 -06:00
Jon Chambers
37e3bcfc3e Move "remove device" logic into AccountsManager 2023-12-05 11:44:58 -05:00
Jon Chambers
4f42c10d60 Disallow sync messages to PNIs 2023-12-05 11:44:27 -05:00
Jonathan Klabunde Tomer
20392a567b Revert "multisend cleanup"
This reverts commit c03249b411.
2023-12-01 14:39:31 -08:00
Jonathan Klabunde Tomer
c03249b411 multisend cleanup 2023-11-30 15:50:36 -08:00
gram-signal
22e6584402 Add auth controller for SVR3 to /v3/backup. 2023-11-30 15:50:21 -08:00
Chris Eager
c18aca9215 Update to the latest version of the spam filter 2023-11-29 16:08:31 -06:00
Jon Chambers
aa23a5422a Don't modify registration IDs from the "set account attributes" endpoint 2023-11-29 16:56:47 -05:00
Jon Chambers
01fde4f9ca Require PNI-associated registration IDs for new devices 2023-11-29 16:56:47 -05:00
Jon Chambers
3980dec123 Revert "Add metrics to check presence of PNI-associated registration IDs"
This reverts commit 1da3f96d10.
2023-11-29 16:56:47 -05:00
Jon Chambers
c97f837f45 Revert "Instrument registration ID changes via the "set account attributes" endpoint"
This reverts commit 8fbc1dac74.
2023-11-29 16:56:47 -05:00
Chris Eager
9c54d2407b Update tests 2023-11-29 15:48:24 -06:00
Chris Eager
a027c4ce1f Conditionally use request.remoteAddr instead of X-Forwarded-For 2023-11-29 15:48:24 -06:00
Chris Eager
b1fd025ea6 Use EpochSecondSerializer for UserRemoteConfigList.serverEpochTime 2023-11-29 15:47:10 -06:00
Jonathan Klabunde Tomer
a05a230085 use Account-specific json serializer when reserving username hash 2023-11-29 13:40:06 -08:00
Jon Chambers
8fbc1dac74 Instrument registration ID changes via the "set account attributes" endpoint 2023-11-29 10:26:41 -05:00
Katherine
f46842c6c9 Validate registration IDs 2023-11-28 15:43:35 -08:00
Ravi Khadiwala
8b95bb0c03 Only run integration tests if integration bucket is configured 2023-11-28 15:10:49 -06:00
ravi-signal
202dd8e92d Add copy endpoint to ArchiveController
Co-authored-by: Jonathan Klabunde Tomer <125505367+jkt-signal@users.noreply.github.com>
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2023-11-28 11:45:41 -06:00
Jon Chambers
1da3f96d10 Add metrics to check presence of PNI-associated registration IDs 2023-11-28 12:26:15 -05:00
Chris Eager
5f6fe4d670 Add NoneHealthResponder to replace default responder 2023-11-27 14:38:47 -06:00
Chris Eager
a74438d1ee Add test for concurrent in-flight outbound messages on WebSocket queue processing
This also elevates the implicit default concurrency (via reactor’s `Queues.SMALL_BUFFER_SIZE`) to be explicit.
2023-11-27 14:37:46 -06:00
Jon Chambers
c8033f875d Create accounts transactionally 2023-11-27 09:57:43 -05:00
Jon Chambers
07c04006df Avoid blocking calls in async account updates 2023-11-27 09:57:43 -05:00
Jon Chambers
521900c048 Always require atomic account creation 2023-11-27 09:52:57 -05:00
Jon Chambers
9069c5abb6 Temporarily make registration challenge notifications "noisy" 2023-11-22 10:25:59 -05:00
Jon Chambers
ff7a5f471b Always use "conserve power" priority for challenge notifications 2023-11-21 17:58:32 -05:00
Ravi Khadiwala
42a47406cc Fix platform tag for push challenge metric 2023-11-21 17:57:32 -05:00
Katherine
de10b6de7b Add metric to count invalid registration ID when setting account attributes 2023-11-20 14:25:27 -08:00
Chris Eager
d6ade0e1ac Send HTTP 408 on idle timeout 2023-11-20 16:24:54 -06:00
Ravi Khadiwala
e04b5e5c9f run integration-test job daily at 19:30 UTC 2023-11-20 15:31:45 -06:00
Ravi Khadiwala
15a6c46d47 use a configured number in integration test 2023-11-20 15:02:53 -06:00
Jonathan Klabunde Tomer
cb1fc734c2 report exceptions in fanned-out sends of multi-recipient messages 2023-11-20 10:46:26 -08:00
Katherine Yen
db7f18aae7 Add missing boolean parameter to getSubscriptionStatus method call 2023-11-16 19:28:10 -05:00
Katherine
7fbc327591 For Braintree, return active subscription status if latest transaction also succeeded 2023-11-16 19:15:46 -05:00
Chris Eager
84b56ae1b2 Upgrade to dropwizard 3.0 2023-11-16 17:57:48 -06:00
Katherine
041aa8639a Enforce story ratelimit 2023-11-16 12:36:43 -05:00
Katherine
216ac72ad0 Use Braintree subscription status alongside transaction status to determine 204 and 402 responses for /v1/subscription/{subscriberId}/receipt_credentials 2023-11-16 12:36:19 -05:00
Jon Chambers
c85ddaeb9c Make DynamoDB client connection pool sizes configurable 2023-11-16 12:29:43 -05:00
Jonathan Klabunde Tomer
e09dec330a Update to the latest version of the spam filter 2023-11-15 17:34:57 -08:00
Jonathan Klabunde Tomer
8f7bae54fe When persisting messages fails due to a full queue in DynamoDB, automatically unlink one device to free up room.
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2023-11-15 17:15:17 -08:00
Jon Chambers
ce60f13320 Don't track IPv4/IPv6 metrics at the application level
We can measure this at the infrastructure level without observing IPs directly.
2023-11-15 17:18:54 -05:00
Jon Chambers
1ac0140666 Remove a couple of now-unused variables 2023-11-15 17:12:08 -05:00
Jon Chambers
6cc8b147a9 Remove unused X-Forwarded-For parameters 2023-11-15 17:09:47 -05:00
Jon Chambers
e078161e2f Consistently use HttpHeaders.X_FORWARDED_FOR 2023-11-15 17:09:36 -05:00
Jonathan Klabunde Tomer
7764185c57 return explicit Response rather than Void from async controllers with no expected body content 2023-11-14 21:57:25 -08:00
Katherine
d4ef2adf0a Remove low priority attempt login notification workaround for old iOS clients 2023-11-13 12:06:55 -05:00
Jonathan Klabunde Tomer
a83378a44e add an option to replace username ciphertext without rotating the link handle 2023-11-13 09:01:54 -08:00
ravi-signal
a4a4204762 Allow re-registered accounts to reclaim their usernames 2023-11-13 10:41:23 -06:00
Jon Chambers
acd1140ef6 Make registration challenge APNs payloads "silent" 2023-11-13 08:39:58 -08:00
Jon Chambers
fbf71c93ff Make rate limit challenge APNs payloads "silent" 2023-11-13 08:39:58 -08:00
Katherine
38bc0c466a Change sepaMaximumEuros field to number in JSON response 2023-11-10 10:16:03 -08:00
Katherine
71e4351743 Add sepaMaximumEuros field to subscription configuration 2023-11-10 09:13:51 -08:00
Katherine
387e4b94b4 Expand charge object on Stripe subscription to surface charge failure information 2023-11-10 09:12:59 -08:00
Katherine
201c76b861 Add charge failure details to /v1/subscription/{subscriberId}/receipt_credentials 402 response 2023-11-08 10:54:14 -08:00
Chris Eager
1c3aa87ca6 Update to the latest version of the spam filter 2023-11-06 10:11:41 -06:00
Sergey Skrobotov
db63ff6b88 gRPC validations 2023-11-03 11:30:48 -07:00
Katherine
115431a486 Un-hardcode payment activation flag 2023-11-03 11:27:34 -07:00
Jonathan Klabunde Tomer
d47ff9b7c7 don't make empty transactions 2023-11-02 16:20:19 -07:00
Chris Eager
b0818148cf Update to the latest version of the spam filter 2023-11-02 11:48:50 -05:00
Chris Eager
2bc4412d66 Encapsulate device ID in ProvisioningAddress 2023-11-02 11:48:10 -05:00
Chris Eager
6a428b4da9 Convert Device.id from long to byte 2023-11-02 11:48:10 -05:00
Jonathan Klabunde Tomer
7299067829 Don't attempt to update PNI PQ prekeys for disabled devices 2023-11-01 16:55:55 -07:00
Chris Eager
5659cb2820 Update to the latest version of the spam filter 2023-11-01 15:53:26 -05:00
Chris Eager
570aa4b9e2 Remove several unused classes 2023-11-01 15:46:10 -05:00
Chris Eager
c4079a3b11 Update to the latest version of the spam filter 2023-11-01 10:07:42 -05:00
Ravi Khadiwala
6b38b538f1 Add ArchiveController
Adds endpoints for creating and managing backup objects with ZK
anonymous credentials.
2023-10-30 14:02:19 -05:00
Chris Eager
ba139dddd8 Use all devices when checking limit 2023-10-30 12:40:06 -05:00
Chris Eager
38b581a231 Update to the latest version of the spam filter 2023-10-27 10:22:50 -05:00
Chris Eager
3c2675b41a Update libphonenumber to 8.13.23 2023-10-27 09:26:40 -05:00
Chris Eager
0f5c62ade5 Set max threads = min threads on command executor services 2023-10-27 09:26:32 -05:00
Jon Chambers
54bc3bce96 Add an authentication-required gRPC service for working with accounts 2023-10-25 14:47:20 -04:00
Jon Chambers
3d92e5b8a9 Explicitly stop and start managed dependencies 2023-10-24 16:50:02 -04:00
Chris Eager
325d145ac3 Update to the latest version of the spam filter 2023-10-24 14:33:31 -05:00
Chris Eager
b0654a416a Update maven plugins 2023-10-24 14:32:55 -05:00
Chris Eager
19930ec2e4 Update dependencies
- AWS: 2.20.130 → 2.21.5
- Braintree: 3.25.0 → 3.27.0
- commons-csv: 2.13.0 → 2.14.0
- dropwizard: 2.1.7 → 2.1.9
- Google libraries BOM: 26.22.0 → 26.25.0
- grpc: 1.56.1 → 1.58.0
- kotlin: 1.9.0 → 1.9.10
- protobuf: 3.23.2 → 3.24.3
- stripe: 23.1.1 → 23.10.0
- junit-pioneer: 2.0.1 → 2.1.0
- firebase-admin: 9.1.1 → 9.2.0
- swagger-jaxrs2: 2.2.8 → 2.2.17
- java-uuid-generator: 4.2.0 → 4.3.0
- log4j: 2.17.0 → 2.21.0
- reactor-bom: 2022.0.10 → 2022.0.12
2023-10-24 14:32:55 -05:00
Jon Chambers
e4de6bf4a7 Only update devices that aren't already disabled 2023-10-24 15:29:03 -04:00
Jon Chambers
21125c2f5a Update to the latest version of the spam filter 2023-10-20 16:38:04 -04:00
Katherine Yen
6f166425fe Fix bank mandate test 2023-10-20 16:19:31 -04:00
Chris Eager
cf2353bcf9 Remove InstrumentedExecutorService wrapping 2023-10-20 15:14:35 -05:00
Jon Chambers
744eb58071 Discard old chunk-based account crawler machinery 2023-10-20 16:09:17 -04:00
Jon Chambers
9d47a6f41f Introduce a reactive push notification feedback processor 2023-10-20 16:09:17 -04:00
Jonathan Klabunde Tomer
4f4c23b12f Update to the latest version of the spam filter 2023-10-20 09:39:46 -07:00
Jonathan Klabunde Tomer
fb02815c27 Update to the latest version of the spam filter 2023-10-20 09:12:37 -07:00
Jonathan Klabunde Tomer
fd19299ae0 Accept a captcha score threshold for challenges from the spam filter 2023-10-20 09:09:22 -07:00
Jon Chambers
9c053e20da Drop Util#isEmpty/Util#nonEmpty in favor of StringUtils 2023-10-20 12:04:15 -04:00
Jon Chambers
19d7b5c65d Drop Util#wait 2023-10-20 12:04:15 -04:00
Jon Chambers
7b9d8829da Remove entirely unused Util methods 2023-10-20 12:04:15 -04:00
Jon Chambers
3505ac498c Update to the latest version of the spam filter 2023-10-20 10:52:43 -04:00
Jon Chambers
f0ab52eb5d Rename "master device" to "primary device" 2023-10-20 10:52:13 -04:00
Jon Chambers
e8cebad27e Avoid modifying original Account instances when constructing JSON for updates 2023-10-20 10:51:50 -04:00
Jon Chambers
6441d5838d Clear username links in the same transaction when clearing username hashes 2023-10-20 10:51:50 -04:00
Jon Chambers
ac0c8b1e9a Introduce a canonical constant for UAK length 2023-10-20 10:50:44 -04:00
Katherine
8ec062fbef Define an endpoint to set the default payment method for iDEAL subscriptions 2023-10-19 10:29:40 -07:00
Katherine
5990a100db Add charge failure details to /v1/subscription/boost/receipt_credential 402 response 2023-10-19 10:21:26 -07:00
Jon Chambers
bc35278684 Drop the old AccountCleaner 2023-10-19 10:34:24 -04:00
Jon Chambers
c3c7329ebb Add a single-shot command for removing expired accounts 2023-10-19 10:34:24 -04:00
Jon Chambers
6fd1c84126 Make command namespace available to subclasses 2023-10-19 10:34:24 -04:00
Jon Chambers
0100f0fcc9 Migrate a username links test from AccountsTest to AccountsManagerUsernameIntegrationTest 2023-10-18 10:20:48 -04:00
Jon Chambers
0cdc32cf65 Really REALLY fix instrumentation for re-registration of recently-deleted accounts 2023-10-18 10:15:03 -04:00
Jon Chambers
601e9eebbd Implement an anonymous account service for looking up accounts 2023-10-18 10:14:52 -04:00
Jon Chambers
eaa868cf06 Add a remote address interceptor to base gRPC tests 2023-10-18 10:14:52 -04:00
Jon Chambers
f55504c665 Add utility methods for rate-limiting by remote address 2023-10-18 10:14:52 -04:00
Katherine Yen
b2ff016cc1 Add back story ratelimiter with counter but do not enforce 2023-10-17 12:22:17 -04:00
Jon Chambers
33b4f17945 Make username-related operations asynchronous 2023-10-17 12:21:52 -04:00
Jon Chambers
e310a3560b Remove unused configuration for the legacy Secure Backup Service 2023-10-17 12:21:14 -04:00
Jon Chambers
162b27323e Fix instrumentation for re-registration of recently-deleted accounts 2023-10-17 12:20:58 -04:00
Jon Chambers
ae976ef8d6 Retire legacy Secure Value Recovery plumbing 2023-10-13 15:32:41 -04:00
Katherine
c6b4e2b71d Support iDEAL 2023-10-12 09:54:05 -07:00
Jon Chambers
33c8bbd0ce Trim stale capabilities from the profiles gRPC service 2023-10-12 12:52:32 -04:00
Jon Chambers
f2a3b8dba4 Treat APNs team/key IDs as secrets so they can change atomically with the key itself 2023-10-12 12:52:13 -04:00
Katherine
207ae6129b Add paymentMethod and paymentProcessing fields to GET /v1/subscription/{subscriberId} endpoint 2023-10-10 09:56:50 -07:00
Katherine
e1aa734c40 Define endpoint to get localized bank mandate text 2023-10-05 09:53:33 -07:00
Jonathan Klabunde Tomer
9b1b03bbfa Update to the latest version of the spam filter 2023-10-05 09:46:27 -07:00
Jon Chambers
bb7e0528c4 Make account deletion an asynchronous operation 2023-10-04 10:44:50 -04:00
Jonathan Klabunde Tomer
010eadcd10 UnlinkDeviceCommand improvements 2023-10-03 15:14:02 -07:00
Katherine
c43e0b54f2 Exclude SEPA_DEBIT as a supported payment method for certain iOS client versions 2023-10-03 11:34:52 -07:00
Chris Eager
6522b74e20 Remove obsolete metrics 2023-10-03 11:42:25 -05:00
Chris Eager
8c7975d89a Clear presence only if the connection’s displacement listener is still present 2023-10-03 11:42:25 -05:00
Chris Eager
407070c9fc Unsubscribe from keyspace notifications only if queue still maps to the listener 2023-10-03 11:42:25 -05:00
Katherine
7821a3cd61 Accommodate PayPal with SEPA changes 2023-09-28 10:28:17 -07:00
Katherine
a00c2fcfdb Support SEPA 2023-09-28 08:26:01 -07:00
Jonathan Klabunde Tomer
9cd21d1326 count ItemCollectionSizeLimitExceededExceptions persisting messages 2023-09-27 10:58:28 -07:00
Jonathan Klabunde Tomer
aaba95f9b8 return null for empty username hash in AccountIdentityResponse 2023-09-27 10:58:04 -07:00
Chris Eager
8d1135a2a3 Refine RegistrationController logic
Local device transfer on iOS uses the `409` status code to prompt the
transfer UI. This needs to happen before sending a `423` and locking
an existing account, since the device transfer
includes the local device database verbatim.
2023-09-25 15:54:31 -05:00
Jon Chambers
f9fabbedce Convert SubscriptionController request/response entities to records 2023-09-25 12:32:49 -07:00
Chris Eager
16012e6ffe Remove obsolete ManagedPeriodicWork 2023-09-25 12:15:17 -07:00
Jon Chambers
d10a132b0c Remove unused methods in SubscriptionController 2023-09-25 12:14:56 -07:00
Sergey Skrobotov
0b3af7d824 gRPC API for external services credentials service 2023-09-25 12:14:49 -07:00
Sergey Skrobotov
d0fdae3df7 Enable header-based auth for WebSocket connections 2023-09-25 12:14:40 -07:00
Ravi Khadiwala
a263611746 editorconfig: keep_simple_classes_in_one_line 2023-09-25 10:10:44 -05:00
Chris Eager
0e989419c6 Add metric for late removal of message availability and displacement listeners 2023-09-19 12:04:24 -05:00
ravi-signal
0fa8276d2d retry hCaptcha errors
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2023-09-14 16:07:35 -05:00
Ravi Khadiwala
b594986241 Set an idle timeout on registration gRPC client 2023-09-14 16:06:49 -05:00
Sergey Skrobotov
9f3ffa3707 gRPC API for payments service 2023-09-14 11:12:00 -07:00
Jonathan Klabunde Tomer
8e598c19dc don't attempt to update KEM prekeys if we have no PQ-enabled devices 2023-09-14 11:11:22 -07:00
Katherine
2601d6e906 Convert some fields on CreateProfileRequest and VersionedProfileResponse to byte arrays 2023-09-13 14:00:03 -07:00
Jon Chambers
de41088051 Update to WireMock 2.35.1 2023-09-13 16:56:15 -04:00
Jon Chambers
f2752b2a02 Update to the latest version of the spam filter 2023-09-13 16:02:46 -04:00
Jon Chambers
f0544fab89 Update recently-deleted accounts table transactionally as part of account mutations 2023-09-13 16:02:19 -04:00
Jon Chambers
1b9bf01ab1 Absorb DeletedAccounts into Accounts 2023-09-13 16:02:19 -04:00
Ravi Khadiwala
9945367fa1 Update to the latest version of the spam filter 2023-09-11 15:19:10 -05:00
Katherine
cbc3887226 Define identity key check endpoint in keys anonymous service 2023-09-11 11:57:00 -07:00
Ravi Khadiwala
c11b74e9c0 Update to the latest version of the spam filter 2023-09-11 13:37:07 -05:00
Jon Chambers
2b764c2abd Don't allow callers to unlink their primary device 2023-09-11 14:29:48 -04:00
Jon Chambers
845fc338d7 Add a (failing) test for removing primary devices from accounts 2023-09-11 14:29:48 -04:00
Sergey Skrobotov
977243ebfd DRY gRPC tests, refactor error mapping 2023-09-08 17:12:08 -07:00
Chris Eager
29ca544c95 Revert "Set suppressCancel=true in Mono.fromFuture"
This reverts commit 8348263fab.
2023-09-07 17:03:33 -05:00
Ravi Khadiwala
94b41d3a2c Fixup default rate limits
A previous refactor left the default rate limits off by a factor of 60.
2023-09-07 16:07:42 -05:00
Chris Eager
92bb783cbb Use static exception instance when a connection is closed 2023-09-07 16:06:16 -05:00
Chris Eager
8348263fab Set suppressCancel=true in Mono.fromFuture 2023-09-07 16:06:03 -05:00
Ravi Khadiwala
48f633de11 Fix type for comparison in integration test 2023-09-07 14:41:29 -05:00
Ravi Khadiwala
b3b9a629f3 Update to the latest version of the spam filter 2023-09-07 11:18:48 -05:00
Ravi Khadiwala
5934b7344a Remove unused captcha configuration 2023-09-07 11:16:32 -05:00
Chris Eager
a9a2e40fed Move onErrorResume to individual sendMessage Mono 2023-09-07 11:15:57 -05:00
Chris Eager
656326355a Invert String.equals() to prevent NullPointerException 2023-09-07 11:14:36 -05:00
Chris Eager
b89e2e5355 Propagate certain subscription processor errors to client responses 2023-09-06 15:57:14 -05:00
Chris Eager
2d187abf13 Handle WebSocket sendMessage errors with onErrorResume 2023-09-06 15:53:01 -05:00
Chris Eager
b701412295 Update maven-wrapper.properties 2023-09-06 15:48:27 -05:00
Jonathan Klabunde Tomer
b4dad81220 Update to the latest version of the spam filter 2023-09-05 13:55:07 -07:00
Jonathan Klabunde Tomer
6bccdad998 Update to the latest version of the spam filter 2023-09-05 10:23:39 -07:00
Chris Eager
ecd6b0174a Add timeouts to crawl chunk join()s 2023-08-31 15:03:19 -05:00
Chris Eager
a1e534a515 Add default request timeout to FaultTolerantHttpClient 2023-08-31 15:03:19 -05:00
Sergey Skrobotov
ebbe19ba63 Add missing copyright headers and reorder some imports 2023-08-30 16:07:53 -07:00
Katherine Yen
6a37b73463 Profile gRPC: Define getExpiringProfileKeyCredential endpoint 2023-08-30 14:56:43 -07:00
Katherine Yen
dd18fcaea2 Profile gRPC: Define getVersionedProfile endpoint 2023-08-30 14:47:11 -07:00
Katherine Yen
5afc058f90 Profile gRPC: Define getUnversionedProfile endpoint 2023-08-30 14:24:43 -07:00
Jon Chambers
5e221fa9a3 Tests for validation of Kyber keys on PNI change/key distribution events
Co-authored-by: Jonathan Klabunde Tomer <jkt@signal.org>
2023-08-30 14:07:33 -07:00
Jon Chambers
0e0cb4d422 Drop the non-normalized account crawler 2023-08-30 13:55:41 -04:00
Jonathan Klabunde Tomer
09f6d60ae9 Update to the latest version of the spam filter 2023-08-29 15:52:42 -07:00
Jonathan Klabunde Tomer
9577d552c6 pass challenge type to rate limit reset listeners 2023-08-29 15:19:49 -07:00
Chris Eager
093f17dce2 Update to stripe-java 23.1.1 2023-08-29 15:18:16 -07:00
Jon Chambers
6089f49b9c Add a gRPC interceptor for getting client addresses 2023-08-29 15:18:06 -07:00
Sergey Skrobotov
cfb910e87e Adding copyright headers to proto files 2023-08-28 14:39:33 -07:00
Ravi Khadiwala
376cffc61d Update to the latest version of the spam filter 2023-08-25 16:49:05 -05:00
Chris Eager
d338ba5152 Convert some KeysController methods return CompletableFutures 2023-08-24 11:59:28 -05:00
Chris Eager
f181397664 Add test for round-trip AccountsManager JSON serialization 2023-08-24 11:18:01 -05:00
Chris Eager
708f23a2ee Remove deprecated identity key and signed pre-key methods 2023-08-24 11:18:01 -05:00
Chris Eager
2d1a979eba Update libphonenumber to 8.13.19 2023-08-24 11:07:18 -05:00
Chris Eager
ee0be92967 Update to the latest version of the spam filter 2023-08-24 11:06:30 -05:00
Chris Eager
7536b75508 Remove unused test fixtures 2023-08-24 11:06:11 -05:00
Jonathan Klabunde Tomer
7237ae6c54 check that pq last-resort prekeys, if submitted, match device list 2023-08-24 09:04:29 -07:00
Sergey Skrobotov
ca05753a3e adding 400 response documentation to the API call 2023-08-23 13:20:07 -07:00
Chris Eager
9ca8503eac Downgrade to stripe-java 22.30.0 2023-08-22 16:31:46 -05:00
Jon Chambers
754f71ce00 Add a gRPC service for working with devices 2023-08-22 16:31:02 -05:00
Jon Chambers
619b05e56c Add utility a method for requiring authentication with the account's primary device 2023-08-22 16:31:02 -05:00
Jon Chambers
8b13826949 Convert DeviceInfo and DeviceInfoList to a record 2023-08-22 16:31:02 -05:00
Jon Chambers
a96ee57c7e Defer asynchronous actions when deriving Mono instances from futures 2023-08-22 16:28:02 -05:00
Jon Chambers
ff1ef90a6d Defer actions taken after rate limit checks 2023-08-22 16:28:02 -05:00
Chris Eager
22905fa8ee Downgrade logstash-logback-encoder to 7.3 2023-08-21 12:44:02 -05:00
Chris Eager
9e218ddd1c Update to the latest version of the spam filter 2023-08-21 11:42:11 -05:00
Chris Eager
6f0462622b Update maven and various plugins 2023-08-21 11:34:08 -05:00
Chris Eager
2f17161163 Update various dependencies 2023-08-21 11:34:08 -05:00
1181 changed files with 88673 additions and 33725 deletions

View File

@@ -158,7 +158,7 @@ ij_java_keep_indents_on_empty_lines = false
ij_java_keep_line_breaks = true
ij_java_keep_multiple_expressions_in_one_line = false
ij_java_keep_simple_blocks_in_one_line = false
ij_java_keep_simple_classes_in_one_line = false
ij_java_keep_simple_classes_in_one_line = true
ij_java_keep_simple_lambdas_in_one_line = false
ij_java_keep_simple_methods_in_one_line = false
ij_java_label_indent_absolute = false

View File

@@ -11,11 +11,11 @@ jobs:
contents: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
- uses: actions/setup-java@5ffc13f4174014e2d4d4572b3d74c3fa61aeb2c2 # v3.11.0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5.0.0
with:
distribution: 'temurin'
java-version: '17'
java-version-file: .java-version
cache: 'maven'
- name: Compile and Build OpenAPI file
run: ./mvnw compile

View File

@@ -1,30 +1,37 @@
name: Integration Tests
on: [workflow_dispatch]
on:
schedule:
- cron: '30 19 * * MON-FRI'
workflow_dispatch:
env:
# This may seem a little redundant, but copying the configuration to an environment variable makes it easier and safer
# to then write its contents to a file
INTEGRATION_TEST_CONFIG: ${{ vars.INTEGRATION_TEST_CONFIG }}
jobs:
build:
if: ${{ vars.INTEGRATION_TEST_CONFIG != '' }}
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
- uses: actions/setup-java@5ffc13f4174014e2d4d4572b3d74c3fa61aeb2c2 # v3.11.0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5.0.0
with:
distribution: 'temurin'
java-version: '17'
java-version-file: .java-version
cache: 'maven'
- uses: aws-actions/configure-aws-credentials@v2
- uses: aws-actions/configure-aws-credentials@b47578312673ae6fa5b5096b330d9fbac3d116df # v5.0.0
name: Configure AWS credentials from Test account
with:
role-to-assume: ${{ vars.AWS_ROLE }}
aws-region: ${{ vars.AWS_REGION }}
- name: Fetch integration utils library
- name: Write integration test configuration
run: |
mkdir -p integration-tests/.libs
mkdir -p integration-tests/src/main/resources
wget -O integration-tests/.libs/software.amazon.awssdk-sso.jar https://repo1.maven.org/maven2/software/amazon/awssdk/sso/2.19.8/sso-2.19.8.jar
aws s3 cp "s3://${{ vars.INTEGRATION_TESTS_BUCKET }}/config-latest.yml" integration-tests/src/main/resources/config.yml
echo "${INTEGRATION_TEST_CONFIG}" > integration-tests/src/main/resources/config.yml
- name: Run and verify integration tests
run: ./mvnw clean compile test-compile failsafe:integration-test failsafe:verify
run: ./mvnw clean compile test-compile failsafe:integration-test failsafe:verify -P aws-sso

View File

@@ -1,6 +1,7 @@
name: Service CI
on:
pull_request:
push:
branches-ignore:
- gh-pages
@@ -8,19 +9,59 @@ on:
jobs:
build:
runs-on: ubuntu-latest
container: ubuntu:22.04
container: ubuntu:24.04
timeout-minutes: 20
services:
foundationdb0:
# Note: this should generally match the version of the FoundationDB SERVER deployed in production; it's okay if
# it's a little behind the CLIENT version.
image: foundationdb/foundationdb:7.3.62
options: --name foundationdb0
foundationdb1:
# Note: this should generally match the version of the FoundationDB SERVER deployed in production; it's okay if
# it's a little behind the CLIENT version.
image: foundationdb/foundationdb:7.3.62
options: --name foundationdb1
steps:
- uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
- name: Set up JDK 17
uses: actions/setup-java@5ffc13f4174014e2d4d4572b3d74c3fa61aeb2c2 # v3.11.0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up JDK
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5.0.0
with:
distribution: 'temurin'
java-version: 17
java-version-file: .java-version
cache: 'maven'
env:
# work around an issue with actions/runner setting an incorrect HOME in containers, which breaks maven caching
# https://github.com/actions/setup-java/issues/356
HOME: /root
- name: Install APT packages
# ca-certificates: required for AWS CRT client
run: |
# Add Docker's official GPG key:
apt update
apt install -y ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
# Add Docker repository to apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
tee /etc/apt/sources.list.d/docker.list > /dev/null
# ca-certificates: required for AWS CRT client
apt update && apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ca-certificates
- name: Configure FoundationDB0 database
run: docker exec foundationdb0 /usr/bin/fdbcli --exec 'configure new single memory'
- name: Configure FoundationDB1 database
run: docker exec foundationdb1 /usr/bin/fdbcli --exec 'configure new single memory'
- name: Download and install FoundationDB client
run: |
./mvnw -e -B -Pexclude-spam-filter clean prepare-package -DskipTests=true
cp service/target/jib-extra/usr/lib/libfdb_c.x86_64.so /usr/lib/libfdb_c.x86_64.so
ldconfig
- name: Build with Maven
run: ./mvnw -e -B verify
run: ./mvnw -e -B clean verify -DfoundationDb.serviceContainerNamePrefix=foundationdb

2
.gitignore vendored
View File

@@ -16,7 +16,6 @@ config/deploy.properties
/service/config/testing.yml
/service/config/deploy.properties
/service/dependency-reduced-pom.xml
.java-version
.opsmanage
put.sh
deployer-staging.properties
@@ -29,3 +28,4 @@ deployer.log
.project
.classpath
.settings
.DS_Store

1
.java-version Normal file
View File

@@ -0,0 +1 @@
temurin-24

View File

@@ -4,6 +4,6 @@
<extension>
<groupId>fr.brouillard.oss</groupId>
<artifactId>jgitver-maven-plugin</artifactId>
<version>1.7.1</version>
<version>1.9.0</version>
</extension>
</extensions>

Binary file not shown.

View File

@@ -14,5 +14,7 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.6/apache-maven-3.8.6-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.1/maven-wrapper-3.1.1.jar
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.11/apache-maven-3.9.11-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.3.4/maven-wrapper-3.3.4.jar
distributionSha256Sum=0d7125e8c91097b36edb990ea5934e6c68b4440eef4ea96510a0f6815e7eeadb
wrapperSha256Sum=4e2fbf6554bc8a4702cdfdd3bef464f423393d784ddbb037216320ce55d5e4e1

View File

@@ -8,9 +8,28 @@ Looking for protocol documentation? Check out the website!
https://signal.org/docs/
Cryptography Notice
How to Build
------------
This project uses [FoundationDB](https://www.foundationdb.org/) and requires the FoundationDB client library to be installed on the host system. With that in place, the server can be built and tested with:
```shell script
$ ./mvnw clean test
```
Security
--------
Security issues should be sent to <a href=mailto:security@signal.org>security@signal.org</a>.
Help
----
We cannot provide direct technical support. Get help running this software in your own environment in our [unofficial community forum][community forum].
Cryptography Notice
-------------------
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software.
BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted.
See <https://www.wassenaar.org/> for more information.
@@ -19,8 +38,10 @@ The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS
The form and manner of this distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
License
---------------------
-------
Copyright 2013-2023 Signal Messenger, LLC
Copyright 2013 Signal Messenger, LLC
Licensed under the AGPLv3: https://www.gnu.org/licenses/agpl-3.0.html
Licensed under the GNU AGPLv3: https://www.gnu.org/licenses/agpl-3.0.html
[community forum]: https://community.signalusers.org

30
TESTING.md Normal file
View File

@@ -0,0 +1,30 @@
# Testing
## Automated tests
The full suite of automated tests can be run using Maven from the project root:
```sh
./mvnw verify
```
## Test server
The service can be run in a feature-limited test mode by running the Maven `integration-test`
goal with the `test-server` profile activated:
```sh
./mvnw integration-test -Ptest-server [-DskipTests=true]
```
This runs [`LocalWhisperServerService`][lwss] with [test configuration][test.yml] and [secrets][test secrets]. External
registration clients are stubbed so that:
- a captcha requirement can be satisfied with `noop.noop.registration.noop`
- any string will be accepted for a phone verification code
[lwss]: service/src/test/java/org/whispersystems/textsecuregcm/LocalWhisperServerService.java
[test.yml]: service/src/test/resources/config/test.yml
[test secrets]: service/src/test/resources/config/test-secrets-bundle.yml

View File

@@ -22,8 +22,8 @@
<plugins>
<plugin>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-maven-plugin</artifactId>
<version>2.2.8</version>
<artifactId>swagger-maven-plugin-jakarta</artifactId>
<version>${swagger.version}</version>
<configuration>
<outputFileName>signal-server-openapi</outputFileName>
<outputPath>${project.build.directory}/openapi</outputPath>

View File

@@ -13,6 +13,7 @@ import io.swagger.v3.jaxrs2.ResolvedParameter;
import io.swagger.v3.jaxrs2.ext.AbstractOpenAPIExtension;
import io.swagger.v3.jaxrs2.ext.OpenAPIExtension;
import io.swagger.v3.oas.models.Components;
import jakarta.ws.rs.Consumes;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import java.util.Iterator;
@@ -20,9 +21,7 @@ import java.util.List;
import java.util.Optional;
import java.util.ServiceLoader;
import java.util.Set;
import javax.ws.rs.Consumes;
import org.whispersystems.textsecuregcm.auth.AuthenticatedAccount;
import org.whispersystems.textsecuregcm.auth.DisabledPermittedAuthenticatedAccount;
import org.whispersystems.textsecuregcm.auth.AuthenticatedDevice;
/**
* One of the extension mechanisms of Swagger Core library (OpenAPI processor) is via custom implementations
@@ -42,10 +41,6 @@ public class OpenApiExtension extends AbstractOpenAPIExtension {
public static final ResolvedParameter OPTIONAL_AUTHENTICATED_ACCOUNT = new ResolvedParameter();
public static final ResolvedParameter DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT = new ResolvedParameter();
public static final ResolvedParameter OPTIONAL_DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT = new ResolvedParameter();
/**
* When parsing endpoint methods, Swagger will treat the first parameter not annotated as header/path/query param
* as a request body (and will ignore other not annotated parameters). In our case, this behavior conflicts with
@@ -67,21 +62,13 @@ public class OpenApiExtension extends AbstractOpenAPIExtension {
if (annotations.stream().anyMatch(a -> a.annotationType().equals(Auth.class))) {
// this is the case of authenticated endpoint,
if (type instanceof SimpleType simpleType
&& simpleType.getRawClass().equals(AuthenticatedAccount.class)) {
&& simpleType.getRawClass().equals(AuthenticatedDevice.class)) {
return AUTHENTICATED_ACCOUNT;
}
if (type instanceof SimpleType simpleType
&& simpleType.getRawClass().equals(DisabledPermittedAuthenticatedAccount.class)) {
return DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT;
}
if (type instanceof SimpleType simpleType
&& isOptionalOfType(simpleType, AuthenticatedAccount.class)) {
&& isOptionalOfType(simpleType, AuthenticatedDevice.class)) {
return OPTIONAL_AUTHENTICATED_ACCOUNT;
}
if (type instanceof SimpleType simpleType
&& isOptionalOfType(simpleType, DisabledPermittedAuthenticatedAccount.class)) {
return OPTIONAL_DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT;
}
}
return super.extractParameters(

View File

@@ -7,9 +7,7 @@ package org.signal.openapi;
import static com.google.common.base.MoreObjects.firstNonNull;
import static org.signal.openapi.OpenApiExtension.AUTHENTICATED_ACCOUNT;
import static org.signal.openapi.OpenApiExtension.DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT;
import static org.signal.openapi.OpenApiExtension.OPTIONAL_AUTHENTICATED_ACCOUNT;
import static org.signal.openapi.OpenApiExtension.OPTIONAL_DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT;
import com.fasterxml.jackson.annotation.JsonView;
import com.google.common.collect.ImmutableList;
@@ -17,11 +15,11 @@ import io.swagger.v3.jaxrs2.Reader;
import io.swagger.v3.jaxrs2.ResolvedParameter;
import io.swagger.v3.oas.models.Operation;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import jakarta.ws.rs.Consumes;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import java.util.Collections;
import java.util.List;
import javax.ws.rs.Consumes;
/**
* One of the extension mechanisms of Swagger Core library (OpenAPI processor) is via custom implementations
@@ -54,13 +52,13 @@ public class OpenApiReader extends Reader {
final ResolvedParameter resolved = super.getParameters(
type, annotations, operation, classConsumes, methodConsumes, jsonViewAnnotation);
if (resolved == AUTHENTICATED_ACCOUNT || resolved == DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT) {
if (resolved == AUTHENTICATED_ACCOUNT) {
operation.setSecurity(ImmutableList.<SecurityRequirement>builder()
.addAll(firstNonNull(operation.getSecurity(), Collections.emptyList()))
.add(new SecurityRequirement().addList(AUTHENTICATED_ACCOUNT_AUTH_SCHEMA))
.build());
}
if (resolved == OPTIONAL_AUTHENTICATED_ACCOUNT || resolved == OPTIONAL_DISABLED_PERMITTED_AUTHENTICATED_ACCOUNT) {
if (resolved == OPTIONAL_AUTHENTICATED_ACCOUNT) {
operation.setSecurity(ImmutableList.<SecurityRequirement>builder()
.addAll(firstNonNull(operation.getSecurity(), Collections.emptyList()))
.add(new SecurityRequirement().addList(AUTHENTICATED_ACCOUNT_AUTH_SCHEMA))

View File

@@ -1,94 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2022 Signal Messenger, LLC
~ SPDX-License-Identifier: AGPL-3.0-only
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>TextSecureServer</artifactId>
<groupId>org.whispersystems.textsecure</groupId>
<version>JGITVER</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>event-logger</artifactId>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-logging</artifactId>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib</artifactId>
<exclusions>
<exclusion>
<groupId>org.jetbrains</groupId>
<!--
depends on an outdated version (13.0) for JDK 6 compatibility, but its safe to override
https://youtrack.jetbrains.com/issue/KT-25047
-->
<artifactId>annotations</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlinx</groupId>
<artifactId>kotlinx-serialization-json</artifactId>
<version>${kotlinx-serialization.version}</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
<testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<version>${kotlin.version}</version>
<executions>
<execution>
<id>compile</id>
<goals>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>test-compile</id>
<goals>
<goal>test-compile</goal>
</goals>
</execution>
</executions>
<configuration>
<compilerPlugins>
<plugin>kotlinx-serialization</plugin>
</compilerPlugins>
</configuration>
<dependencies>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-serialization</artifactId>
<version>${kotlin.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<configuration>
<!-- we don't want jib to execute on this module -->
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>

View File

@@ -1,40 +0,0 @@
/*
* Copyright 2022 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.signal.event
import java.util.Collections
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass
val module = SerializersModule {
polymorphic(Event::class) {
subclass(RemoteConfigSetEvent::class)
subclass(RemoteConfigDeleteEvent::class)
}
}
val jsonFormat = Json { serializersModule = module }
sealed interface Event
@Serializable
data class RemoteConfigSetEvent(
val identity: String,
val name: String,
val percentage: Int,
val defaultValue: String? = null,
val value: String? = null,
val hashKey: String? = null,
val uuids: Collection<String> = Collections.emptyList(),
) : Event
@Serializable
data class RemoteConfigDeleteEvent(
val identity: String,
val name: String,
) : Event

View File

@@ -1,41 +0,0 @@
/*
* Copyright 2022 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.signal.event
import com.google.cloud.logging.LogEntry
import com.google.cloud.logging.Logging
import com.google.cloud.logging.MonitoredResourceUtil
import com.google.cloud.logging.Payload.JsonPayload
import com.google.cloud.logging.Severity
import com.google.protobuf.Struct
import com.google.protobuf.util.JsonFormat
import kotlinx.serialization.encodeToString
interface AdminEventLogger {
fun logEvent(event: Event, labels: Map<String, String>?)
fun logEvent(event: Event) = logEvent(event, null)
}
class NoOpAdminEventLogger : AdminEventLogger {
override fun logEvent(event: Event, labels: Map<String, String>?) {}
}
class GoogleCloudAdminEventLogger(private val logging: Logging, private val projectId: String, private val logName: String) : AdminEventLogger {
override fun logEvent(event: Event, labels: Map<String, String>?) {
val structBuilder = Struct.newBuilder()
JsonFormat.parser().merge(jsonFormat.encodeToString(event), structBuilder)
val struct = structBuilder.build()
val logEntryBuilder = LogEntry.newBuilder(JsonPayload.of(struct))
.setLogName(logName)
.setSeverity(Severity.NOTICE)
.setResource(MonitoredResourceUtil.getResource(projectId, "project"));
if (labels != null) {
logEntryBuilder.setLabels(labels);
}
logging.write(listOf(logEntryBuilder.build()))
}
}

View File

@@ -1,35 +0,0 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.signal.event
import com.google.cloud.logging.Logging
import com.google.cloud.logging.LoggingOptions
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertDoesNotThrow
import org.mockito.Mockito.mock
class GoogleCloudAdminEventLoggerTest {
@Test
fun logEvent() {
val logging = mock(Logging::class.java)
val logger = GoogleCloudAdminEventLogger(logging, "my-project", "test")
val event = RemoteConfigDeleteEvent("token", "test")
logger.logEvent(event)
}
@Test
fun testGetService() {
assertDoesNotThrow {
// This is a canary for version conflicts between the cloud logging library and protobuf-java
LoggingOptions.newBuilder()
.setProjectId("test")
.build()
.getService()
}
}
}

View File

@@ -28,7 +28,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<configuration>
<excludes>
<exclude>**</exclude>
@@ -39,7 +38,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<additionalClasspathElements>
<additionalClasspathElement>${project.basedir}/.libs/software.amazon.awssdk-sso.jar</additionalClasspathElement>
@@ -59,4 +57,17 @@
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>aws-sso</id>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sso</artifactId>
</dependency>
</dependencies>
</profile>
</profiles>
</project>

View File

@@ -14,7 +14,6 @@ import com.fasterxml.jackson.databind.SerializerProvider;
import java.io.IOException;
import java.util.Base64;
import org.signal.libsignal.protocol.IdentityKey;
import org.signal.libsignal.protocol.ecc.Curve;
import org.signal.libsignal.protocol.ecc.ECPublicKey;
public final class Codecs {
@@ -84,7 +83,7 @@ public final class Codecs {
public static class ECPublicKeyDeserializer extends Base64BasedDeserializer<ECPublicKey> {
public ECPublicKeyDeserializer() {
super(bytes -> Curve.decodePoint(bytes, 0));
super(ECPublicKey::new);
}
}

View File

@@ -8,18 +8,19 @@ package org.signal.integration;
import java.time.Clock;
import java.time.Duration;
import java.util.Optional;
import java.util.UUID;
import java.util.concurrent.CompletableFuture;
import org.signal.integration.config.Config;
import org.whispersystems.textsecuregcm.metrics.NoopAwsSdkMetricPublisher;
import org.whispersystems.textsecuregcm.registration.VerificationSession;
import org.whispersystems.textsecuregcm.storage.PhoneNumberIdentifiers;
import org.whispersystems.textsecuregcm.storage.RegistrationRecoveryPasswords;
import org.whispersystems.textsecuregcm.storage.RegistrationRecoveryPasswordsManager;
import org.whispersystems.textsecuregcm.storage.VerificationSessionManager;
import org.whispersystems.textsecuregcm.storage.VerificationSessions;
import org.whispersystems.textsecuregcm.util.DynamoDbFromConfig;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
public class IntegrationTools {
@@ -27,39 +28,41 @@ public class IntegrationTools {
private final VerificationSessionManager verificationSessionManager;
private final PhoneNumberIdentifiers phoneNumberIdentifiers;
public static IntegrationTools create(final Config config) {
final AwsCredentialsProvider credentialsProvider = DefaultCredentialsProvider.builder().build();
final DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbFromConfig.asyncClient(
config.dynamoDbClientConfiguration(),
credentialsProvider);
final DynamoDbClient dynamoDbClient = DynamoDbFromConfig.client(
config.dynamoDbClientConfiguration(),
credentialsProvider);
final DynamoDbAsyncClient dynamoDbAsyncClient =
config.dynamoDbClient().buildAsyncClient(credentialsProvider, new NoopAwsSdkMetricPublisher());
final RegistrationRecoveryPasswords registrationRecoveryPasswords = new RegistrationRecoveryPasswords(
config.dynamoDbTables().registrationRecovery(), Duration.ofDays(1), dynamoDbClient, dynamoDbAsyncClient);
config.dynamoDbTables().registrationRecovery(), Duration.ofDays(1), dynamoDbAsyncClient, Clock.systemUTC());
final VerificationSessions verificationSessions = new VerificationSessions(
dynamoDbAsyncClient, config.dynamoDbTables().verificationSessions(), Clock.systemUTC());
return new IntegrationTools(
new RegistrationRecoveryPasswordsManager(registrationRecoveryPasswords),
new VerificationSessionManager(verificationSessions)
new VerificationSessionManager(verificationSessions),
new PhoneNumberIdentifiers(dynamoDbAsyncClient, config.dynamoDbTables().phoneNumberIdentifiers())
);
}
private IntegrationTools(
final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager,
final VerificationSessionManager verificationSessionManager) {
final VerificationSessionManager verificationSessionManager,
final PhoneNumberIdentifiers phoneNumberIdentifiers) {
this.registrationRecoveryPasswordsManager = registrationRecoveryPasswordsManager;
this.verificationSessionManager = verificationSessionManager;
this.phoneNumberIdentifiers = phoneNumberIdentifiers;
}
public CompletableFuture<Void> populateRecoveryPassword(final String e164, final byte[] password) {
return registrationRecoveryPasswordsManager.storeForCurrentNumber(e164, password);
public CompletableFuture<Void> populateRecoveryPassword(final String phoneNumber, final byte[] password) {
return phoneNumberIdentifiers
.getPhoneNumberIdentifier(phoneNumber)
.thenCompose(pni -> registrationRecoveryPasswordsManager.store(pni, password));
}
public CompletableFuture<Optional<String>> peekVerificationSessionPushChallenge(final String sessionId) {

View File

@@ -10,6 +10,9 @@ import static java.util.Objects.requireNonNull;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.google.common.io.Resources;
import com.google.common.net.HttpHeaders;
import io.dropwizard.configuration.ConfigurationValidationException;
import io.dropwizard.jersey.validation.Validators;
import jakarta.validation.ConstraintViolation;
import java.io.IOException;
import java.lang.invoke.MethodHandles;
import java.net.URI;
@@ -17,20 +20,20 @@ import java.net.URL;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.Executors;
import org.apache.commons.lang3.RandomUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.Validate;
import org.apache.commons.lang3.tuple.Pair;
import org.signal.integration.config.Config;
import org.signal.libsignal.protocol.IdentityKey;
import org.signal.libsignal.protocol.ecc.Curve;
import org.signal.libsignal.protocol.ecc.ECKeyPair;
import org.signal.libsignal.protocol.ecc.ECPublicKey;
import org.signal.libsignal.protocol.kem.KEMKeyPair;
@@ -38,15 +41,16 @@ import org.signal.libsignal.protocol.kem.KEMKeyType;
import org.signal.libsignal.protocol.kem.KEMPublicKey;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.configuration.CircuitBreakerConfiguration;
import org.whispersystems.textsecuregcm.entities.AccountAttributes;
import org.whispersystems.textsecuregcm.entities.AccountIdentityResponse;
import org.whispersystems.textsecuregcm.entities.DeviceActivationRequest;
import org.whispersystems.textsecuregcm.entities.ECSignedPreKey;
import org.whispersystems.textsecuregcm.entities.KEMSignedPreKey;
import org.whispersystems.textsecuregcm.entities.RegistrationRequest;
import org.whispersystems.textsecuregcm.http.FaultTolerantHttpClient;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.util.HeaderUtils;
import org.whispersystems.textsecuregcm.util.HttpUtils;
import org.whispersystems.textsecuregcm.util.SystemMapper;
public final class Operations {
@@ -67,61 +71,28 @@ public final class Operations {
}
public static TestUser newRegisteredUser(final String number) {
final byte[] registrationPassword = RandomUtils.nextBytes(32);
final String accountPassword = Base64.getEncoder().encodeToString(RandomUtils.nextBytes(32));
final byte[] registrationPassword = populateRandomRecoveryPassword(number);
final String accountPassword = Base64.getEncoder().encodeToString(randomBytes(32));
final TestUser user = TestUser.create(number, accountPassword, registrationPassword);
final AccountAttributes accountAttributes = user.accountAttributes();
INTEGRATION_TOOLS.populateRecoveryPassword(number, registrationPassword).join();
// register account
final RegistrationRequest registrationRequest = new RegistrationRequest(
null, registrationPassword, accountAttributes, true, false,
Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty(), Optional.empty());
final AccountIdentityResponse registrationResponse = apiPost("/v1/registration", registrationRequest)
.authorized(number, accountPassword)
.executeExpectSuccess(AccountIdentityResponse.class);
user.setAciUuid(registrationResponse.uuid());
user.setPniUuid(registrationResponse.pni());
// upload pre-key
final TestUser.PreKeySetPublicView preKeySetPublicView = user.preKeys(Device.MASTER_ID, false);
apiPut("/v2/keys", preKeySetPublicView)
.authorized(user, Device.MASTER_ID)
.executeExpectSuccess();
return user;
}
public static TestUser newRegisteredUserAtomic(final String number) {
final byte[] registrationPassword = RandomUtils.nextBytes(32);
final String accountPassword = Base64.getEncoder().encodeToString(RandomUtils.nextBytes(32));
final TestUser user = TestUser.create(number, accountPassword, registrationPassword);
final AccountAttributes accountAttributes = user.accountAttributes();
INTEGRATION_TOOLS.populateRecoveryPassword(number, registrationPassword).join();
final ECKeyPair aciIdentityKeyPair = Curve.generateKeyPair();
final ECKeyPair pniIdentityKeyPair = Curve.generateKeyPair();
final ECKeyPair aciIdentityKeyPair = ECKeyPair.generate();
final ECKeyPair pniIdentityKeyPair = ECKeyPair.generate();
// register account
final RegistrationRequest registrationRequest = new RegistrationRequest(null,
registrationPassword,
accountAttributes,
true,
true,
Optional.of(new IdentityKey(aciIdentityKeyPair.getPublicKey())),
Optional.of(new IdentityKey(pniIdentityKeyPair.getPublicKey())),
Optional.of(generateSignedECPreKey(1, aciIdentityKeyPair)),
Optional.of(generateSignedECPreKey(2, pniIdentityKeyPair)),
Optional.of(generateSignedKEMPreKey(3, aciIdentityKeyPair)),
Optional.of(generateSignedKEMPreKey(4, pniIdentityKeyPair)),
Optional.empty(),
Optional.empty());
new IdentityKey(aciIdentityKeyPair.getPublicKey()),
new IdentityKey(pniIdentityKeyPair.getPublicKey()),
new DeviceActivationRequest(generateSignedECPreKey(1, aciIdentityKeyPair),
generateSignedECPreKey(2, pniIdentityKeyPair),
generateSignedKEMPreKey(3, aciIdentityKeyPair),
generateSignedKEMPreKey(4, pniIdentityKeyPair),
Optional.empty(),
Optional.empty()));
final AccountIdentityResponse registrationResponse = apiPost("/v1/registration", registrationRequest)
.authorized(number, accountPassword)
@@ -133,6 +104,14 @@ public final class Operations {
return user;
}
public record PrescribedVerificationNumber(String number, String verificationCode) {}
public static PrescribedVerificationNumber prescribedVerificationNumber() {
return new PrescribedVerificationNumber(
CONFIG.prescribedRegistrationNumber(),
CONFIG.prescribedRegistrationCode());
}
public static void deleteUser(final TestUser user) {
apiDelete("/v1/accounts/me").authorized(user).executeExpectSuccess();
}
@@ -142,6 +121,13 @@ public final class Operations {
.orElseThrow(() -> new RuntimeException("push challenge not found for the verification session"));
}
public static byte[] populateRandomRecoveryPassword(final String number) {
final byte[] recoveryPassword = randomBytes(32);
INTEGRATION_TOOLS.populateRecoveryPassword(number, recoveryPassword).join();
return recoveryPassword;
}
public static <T> T sendEmptyRequestAuthenticated(
final String endpoint,
final String method,
@@ -180,6 +166,12 @@ public final class Operations {
}
}
private static byte[] randomBytes(int numBytes) {
final byte[] bytes = new byte[numBytes];
new SecureRandom().nextBytes(bytes);
return bytes;
}
public static RequestBuilder apiGet(final String endpoint) {
return new RequestBuilder(HttpRequest.newBuilder().GET(), endpoint);
}
@@ -233,10 +225,10 @@ public final class Operations {
}
public RequestBuilder authorized(final TestUser user) {
return authorized(user, Device.MASTER_ID);
return authorized(user, Device.PRIMARY_ID);
}
public RequestBuilder authorized(final TestUser user, final long deviceId) {
public RequestBuilder authorized(final TestUser user, final byte deviceId) {
final String username = "%s.%d".formatted(user.aciUuid().toString(), deviceId);
return authorized(username, user.accountPassword());
}
@@ -263,7 +255,7 @@ public final class Operations {
public Pair<Integer, Void> executeExpectSuccess() {
final Pair<Integer, Void> execute = execute();
Validate.isTrue(
execute.getLeft() >= 200 && execute.getLeft() < 300,
HttpUtils.isSuccessfulResponse(execute.getLeft()),
"Unexpected response code: %d",
execute.getLeft());
return execute;
@@ -271,6 +263,10 @@ public final class Operations {
public <T> T executeExpectSuccess(final Class<T> expectedType) {
final Pair<Integer, T> execute = execute(expectedType);
Validate.isTrue(
HttpUtils.isSuccessfulResponse(execute.getLeft()),
"Unexpected response code: %d : %s",
execute.getLeft(), execute.getRight());
return requireNonNull(execute.getRight());
}
@@ -309,11 +305,7 @@ public final class Operations {
private static FaultTolerantHttpClient buildClient() {
try {
return FaultTolerantHttpClient.newBuilder()
.withName("integration-test")
.withExecutor(Executors.newFixedThreadPool(16))
.withRetryExecutor(Executors.newSingleThreadScheduledExecutor())
.withCircuitBreaker(new CircuitBreakerConfiguration())
return FaultTolerantHttpClient.newBuilder("integration-test", Executors.newFixedThreadPool(16))
.withTrustedServerCertificates(CONFIG.rootCert())
.build();
} catch (final CertificateException e) {
@@ -324,21 +316,29 @@ public final class Operations {
private static Config loadConfigFromClasspath(final String filename) {
try {
final URL configFileUrl = Resources.getResource(filename);
return SystemMapper.yamlMapper().readValue(Resources.toByteArray(configFileUrl), Config.class);
} catch (final IOException e) {
final Config config = SystemMapper.yamlMapper().readValue(Resources.toByteArray(configFileUrl), Config.class);
final Set<ConstraintViolation<Config>> constraintViolations = Validators.newValidator().validate(config);
if (!constraintViolations.isEmpty()) {
throw new ConfigurationValidationException(filename, constraintViolations);
}
return config;
} catch (final Exception e) {
throw new RuntimeException(e);
}
}
private static ECSignedPreKey generateSignedECPreKey(long id, final ECKeyPair identityKeyPair) {
final ECPublicKey pubKey = Curve.generateKeyPair().getPublicKey();
final byte[] sig = identityKeyPair.getPrivateKey().calculateSignature(pubKey.serialize());
return new ECSignedPreKey(id, pubKey, sig);
public static ECSignedPreKey generateSignedECPreKey(final long id, final ECKeyPair identityKeyPair) {
final ECPublicKey pubKey = ECKeyPair.generate().getPublicKey();
final byte[] signature = identityKeyPair.getPrivateKey().calculateSignature(pubKey.serialize());
return new ECSignedPreKey(id, pubKey, signature);
}
private static KEMSignedPreKey generateSignedKEMPreKey(long id, final ECKeyPair identityKeyPair) {
public static KEMSignedPreKey generateSignedKEMPreKey(final long id, final ECKeyPair identityKeyPair) {
final KEMPublicKey pubKey = KEMKeyPair.generate(KEMKeyType.KYBER_1024).getPublicKey();
final byte[] sig = identityKeyPair.getPrivateKey().calculateSignature(pubKey.serialize());
return new KEMSignedPreKey(id, pubKey, sig);
final byte[] signature = identityKeyPair.getPrivateKey().calculateSignature(pubKey.serialize());
return new KEMSignedPreKey(id, pubKey, signature);
}
}

View File

@@ -9,20 +9,18 @@ import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.commons.lang3.tuple.Pair;
import org.signal.libsignal.protocol.IdentityKeyPair;
import org.signal.libsignal.protocol.InvalidKeyException;
import org.signal.libsignal.protocol.ecc.Curve;
import org.signal.libsignal.protocol.ecc.ECKeyPair;
import org.signal.libsignal.protocol.state.SignedPreKeyRecord;
public class TestDevice {
private final long deviceId;
private final byte deviceId;
private final Map<Integer, Pair<IdentityKeyPair, SignedPreKeyRecord>> signedPreKeys = new ConcurrentHashMap<>();
public static TestDevice create(
final long deviceId,
final byte deviceId,
final IdentityKeyPair aciIdentityKeyPair,
final IdentityKeyPair pniIdentityKeyPair) {
final TestDevice device = new TestDevice(deviceId);
@@ -31,11 +29,11 @@ public class TestDevice {
return device;
}
public TestDevice(final long deviceId) {
public TestDevice(final byte deviceId) {
this.deviceId = deviceId;
}
public long deviceId() {
public byte deviceId() {
return deviceId;
}
@@ -50,15 +48,11 @@ public class TestDevice {
}
public SignedPreKeyRecord addSignedPreKey(final IdentityKeyPair identity) {
try {
final int nextId = signedPreKeys.keySet().stream().mapToInt(k -> k + 1).max().orElse(0);
final ECKeyPair keyPair = Curve.generateKeyPair();
final byte[] signature = Curve.calculateSignature(identity.getPrivateKey(), keyPair.getPublicKey().serialize());
final SignedPreKeyRecord signedPreKeyRecord = new SignedPreKeyRecord(nextId, System.currentTimeMillis(), keyPair, signature);
signedPreKeys.put(nextId, Pair.of(identity, signedPreKeyRecord));
return signedPreKeyRecord;
} catch (InvalidKeyException e) {
throw new RuntimeException(e);
}
final int nextId = signedPreKeys.keySet().stream().mapToInt(k -> k + 1).max().orElse(0);
final ECKeyPair keyPair = ECKeyPair.generate();
final byte[] signature = keyPair.getPrivateKey().calculateSignature(keyPair.getPublicKey().serialize());
final SignedPreKeyRecord signedPreKeyRecord = new SignedPreKeyRecord(nextId, System.currentTimeMillis(), keyPair, signature);
signedPreKeys.put(nextId, Pair.of(identity, signedPreKeyRecord));
return signedPreKeyRecord;
}
}

View File

@@ -9,17 +9,21 @@ import static java.util.Objects.requireNonNull;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.security.SecureRandom;
import java.nio.charset.StandardCharsets;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.commons.lang3.RandomUtils;
import org.signal.libsignal.protocol.IdentityKey;
import org.signal.libsignal.protocol.IdentityKeyPair;
import org.signal.libsignal.protocol.InvalidKeyException;
import org.signal.libsignal.protocol.ecc.ECPublicKey;
import org.signal.libsignal.protocol.state.SignedPreKeyRecord;
import org.signal.libsignal.protocol.util.KeyHelper;
import org.whispersystems.textsecuregcm.auth.UnidentifiedAccessUtil;
import org.whispersystems.textsecuregcm.entities.AccountAttributes;
import org.whispersystems.textsecuregcm.storage.Device;
@@ -27,9 +31,11 @@ public class TestUser {
private final int registrationId;
private final int pniRegistrationId;
private final IdentityKeyPair aciIdentityKey;
private final Map<Long, TestDevice> devices = new ConcurrentHashMap<>();
private final Map<Byte, TestDevice> devices = new ConcurrentHashMap<>();
private final byte[] unidentifiedAccessKey;
@@ -53,11 +59,14 @@ public class TestUser {
final IdentityKeyPair pniIdentityKey = IdentityKeyPair.generate();
// registration id
final int registrationId = KeyHelper.generateRegistrationId(false);
final int pniRegistrationId = KeyHelper.generateRegistrationId(false);
// uak
final byte[] unidentifiedAccessKey = RandomUtils.nextBytes(16);
final byte[] unidentifiedAccessKey = new byte[UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH];
new SecureRandom().nextBytes(unidentifiedAccessKey);
return new TestUser(
registrationId,
pniRegistrationId,
aciIdentityKey,
phoneNumber,
pniIdentityKey,
@@ -68,6 +77,7 @@ public class TestUser {
public TestUser(
final int registrationId,
final int pniRegistrationId,
final IdentityKeyPair aciIdentityKey,
final String phoneNumber,
final IdentityKeyPair pniIdentityKey,
@@ -75,13 +85,14 @@ public class TestUser {
final String accountPassword,
final byte[] registrationPassword) {
this.registrationId = registrationId;
this.pniRegistrationId = pniRegistrationId;
this.aciIdentityKey = aciIdentityKey;
this.phoneNumber = phoneNumber;
this.pniIdentityKey = pniIdentityKey;
this.unidentifiedAccessKey = unidentifiedAccessKey;
this.accountPassword = accountPassword;
this.registrationPassword = registrationPassword;
devices.put(Device.MASTER_ID, TestDevice.create(Device.MASTER_ID, aciIdentityKey, pniIdentityKey));
devices.put(Device.PRIMARY_ID, TestDevice.create(Device.PRIMARY_ID, aciIdentityKey, pniIdentityKey));
}
public int registrationId() {
@@ -117,7 +128,7 @@ public class TestUser {
}
public AccountAttributes accountAttributes() {
return new AccountAttributes(true, registrationId, "", "", true, new Device.DeviceCapabilities(false, false, false, false))
return new AccountAttributes(true, registrationId, pniRegistrationId, "".getBytes(StandardCharsets.UTF_8), "", true, Set.of())
.withUnidentifiedAccessKey(unidentifiedAccessKey)
.withRecoveryPassword(registrationPassword);
}
@@ -146,21 +157,25 @@ public class TestUser {
this.registrationPassword = registrationPassword;
}
public PreKeySetPublicView preKeys(final long deviceId, final boolean pni) {
public PreKeySetPublicView preKeys(final byte deviceId, final boolean pni) {
final IdentityKeyPair identity = pni
? pniIdentityKey
: aciIdentityKey;
final TestDevice device = requireNonNull(devices.get(deviceId));
final SignedPreKeyRecord signedPreKeyRecord = device.latestSignedPreKey(identity);
return new PreKeySetPublicView(
Collections.emptyList(),
identity.getPublicKey(),
new SignedPreKeyPublicView(
signedPreKeyRecord.getId(),
signedPreKeyRecord.getKeyPair().getPublicKey(),
signedPreKeyRecord.getSignature()
)
);
try {
return new PreKeySetPublicView(
Collections.emptyList(),
identity.getPublicKey(),
new SignedPreKeyPublicView(
signedPreKeyRecord.getId(),
signedPreKeyRecord.getKeyPair().getPublicKey(),
signedPreKeyRecord.getSignature()
)
);
} catch (InvalidKeyException e) {
throw new RuntimeException(e);
}
}
public record SignedPreKeyPublicView(

View File

@@ -5,10 +5,15 @@
package org.signal.integration.config;
import org.whispersystems.textsecuregcm.configuration.DynamoDbClientConfiguration;
import jakarta.validation.Valid;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.NotNull;
import org.whispersystems.textsecuregcm.configuration.DynamoDbClientFactory;
public record Config(String domain,
String rootCert,
DynamoDbClientConfiguration dynamoDbClientConfiguration,
DynamoDbTables dynamoDbTables) {
public record Config(@NotBlank String domain,
@NotBlank String rootCert,
@NotNull @Valid DynamoDbClientFactory dynamoDbClient,
@NotNull @Valid DynamoDbTables dynamoDbTables,
@NotBlank String prescribedRegistrationNumber,
@NotBlank String prescribedRegistrationCode) {
}

View File

@@ -5,6 +5,9 @@
package org.signal.integration.config;
public record DynamoDbTables(String registrationRecovery,
String verificationSessions) {
import jakarta.validation.constraints.NotBlank;
public record DynamoDbTables(@NotBlank String registrationRecovery,
@NotBlank String verificationSessions,
@NotBlank String phoneNumberIdentifiers) {
}

View File

@@ -6,27 +6,34 @@
package org.signal.integration;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.http.HttpStatus;
import org.junit.jupiter.api.Test;
import org.signal.libsignal.protocol.IdentityKey;
import org.signal.libsignal.protocol.ecc.ECKeyPair;
import org.signal.libsignal.usernames.BaseUsernameException;
import org.signal.libsignal.usernames.Username;
import org.whispersystems.textsecuregcm.entities.AccountIdentifierResponse;
import org.whispersystems.textsecuregcm.entities.AccountIdentityResponse;
import org.whispersystems.textsecuregcm.entities.ChangeNumberRequest;
import org.whispersystems.textsecuregcm.entities.ConfirmUsernameHashRequest;
import org.whispersystems.textsecuregcm.entities.EncryptedUsername;
import org.whispersystems.textsecuregcm.entities.ReserveUsernameHashRequest;
import org.whispersystems.textsecuregcm.entities.ReserveUsernameHashResponse;
import org.whispersystems.textsecuregcm.entities.UsernameHashResponse;
import org.whispersystems.textsecuregcm.identity.AciServiceIdentifier;
import org.whispersystems.textsecuregcm.storage.Device;
public class AccountTest {
@Test
public void testCreateAccount() throws Exception {
public void testCreateAccount() {
final TestUser user = Operations.newRegisteredUser("+19995550101");
try {
final Pair<Integer, AccountIdentityResponse> execute = Operations.apiGet("/v1/accounts/whoami")
@@ -39,8 +46,8 @@ public class AccountTest {
}
@Test
public void testCreateAccountAtomic() throws Exception {
final TestUser user = Operations.newRegisteredUserAtomic("+19995550201");
public void testCreateAccountAtomic() {
final TestUser user = Operations.newRegisteredUser("+19995550201");
try {
final Pair<Integer, AccountIdentityResponse> execute = Operations.apiGet("/v1/accounts/whoami")
.authorized(user)
@@ -51,6 +58,33 @@ public class AccountTest {
}
}
@Test
public void changePhoneNumber() {
final TestUser user = Operations.newRegisteredUser("+19995550301");
final String targetNumber = "+19995550302";
final ECKeyPair pniIdentityKeyPair = ECKeyPair.generate();
final ChangeNumberRequest changeNumberRequest = new ChangeNumberRequest(null,
Operations.populateRandomRecoveryPassword(targetNumber),
targetNumber,
null,
new IdentityKey(pniIdentityKeyPair.getPublicKey()),
Collections.emptyList(),
Map.of(Device.PRIMARY_ID, Operations.generateSignedECPreKey(1, pniIdentityKeyPair)),
Map.of(Device.PRIMARY_ID, Operations.generateSignedKEMPreKey(2, pniIdentityKeyPair)),
Map.of(Device.PRIMARY_ID, 17));
final AccountIdentityResponse accountIdentityResponse =
Operations.apiPut("/v2/accounts/number", changeNumberRequest)
.authorized(user)
.executeExpectSuccess(AccountIdentityResponse.class);
assertEquals(user.aciUuid(), accountIdentityResponse.uuid());
assertNotEquals(user.pniUuid(), accountIdentityResponse.pni());
assertEquals(targetNumber, accountIdentityResponse.number());
}
@Test
public void testUsernameOperations() throws Exception {
final TestUser user = Operations.newRegisteredUser("+19995550102");
@@ -107,7 +141,7 @@ public class AccountTest {
final AccountIdentifierResponse accountIdentifierResponse = Operations
.apiGet("/v1/accounts/username_hash/" + Base64.getUrlEncoder().encodeToString(reservedHash))
.executeExpectSuccess(AccountIdentifierResponse.class);
assertEquals(user.aciUuid(), accountIdentifierResponse.uuid());
assertEquals(new AciServiceIdentifier(user.aciUuid()), accountIdentifierResponse.uuid());
// try authorized
Operations
.apiGet("/v1/accounts/username_hash/" + Base64.getUrlEncoder().encodeToString(reservedHash))

View File

@@ -8,11 +8,9 @@ package org.signal.integration;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.List;
import org.apache.commons.lang3.tuple.Pair;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.junit.jupiter.api.Test;
import org.whispersystems.textsecuregcm.entities.IncomingMessage;
import org.whispersystems.textsecuregcm.entities.IncomingMessageList;
import org.whispersystems.textsecuregcm.entities.OutgoingMessageEntityList;
@@ -21,27 +19,17 @@ import org.whispersystems.textsecuregcm.storage.Device;
public class MessagingTest {
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void testSendMessageUnsealed(final boolean atomicAccountCreation) throws Exception {
final TestUser userA;
final TestUser userB;
if (atomicAccountCreation) {
userA = Operations.newRegisteredUserAtomic("+19995550102");
userB = Operations.newRegisteredUserAtomic("+19995550103");
} else {
userA = Operations.newRegisteredUser("+19995550104");
userB = Operations.newRegisteredUser("+19995550105");
}
@Test
public void testSendMessageUnsealed() {
final TestUser userA = Operations.newRegisteredUser("+19995550102");
final TestUser userB = Operations.newRegisteredUser("+19995550103");
try {
final byte[] expectedContent = "Hello, World!".getBytes(StandardCharsets.UTF_8);
final String contentBase64 = Base64.getEncoder().encodeToString(expectedContent);
final IncomingMessage message = new IncomingMessage(1, Device.MASTER_ID, userB.registrationId(), contentBase64);
final IncomingMessage message = new IncomingMessage(1, Device.PRIMARY_ID, userB.registrationId(), expectedContent);
final IncomingMessageList messages = new IncomingMessageList(List.of(message), false, true, System.currentTimeMillis());
final Pair<Integer, SendMessageResponse> sendMessage = Operations
Operations
.apiPut("/v1/messages/%s".formatted(userB.aciUuid().toString()), messages)
.authorized(userA)
.execute(SendMessageResponse.class);
@@ -50,7 +38,7 @@ public class MessagingTest {
.authorized(userB)
.execute(OutgoingMessageEntityList.class);
final byte[] actualContent = receiveMessages.getRight().messages().get(0).content();
final byte[] actualContent = receiveMessages.getRight().messages().getFirst().content();
assertArrayEquals(expectedContent, actualContent);
} finally {
Operations.deleteUser(userA);

View File

@@ -5,6 +5,7 @@
package org.signal.integration;
import io.micrometer.common.util.StringUtils;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.whispersystems.textsecuregcm.entities.CreateVerificationSessionRequest;
@@ -19,13 +20,18 @@ public class RegistrationTest {
public void testRegistration() throws Exception {
final UpdateVerificationSessionRequest originalRequest = new UpdateVerificationSessionRequest(
"test", UpdateVerificationSessionRequest.PushTokenType.FCM, null, null, null, null);
final CreateVerificationSessionRequest input = new CreateVerificationSessionRequest("+19995550102", originalRequest);
final Operations.PrescribedVerificationNumber params = Operations.prescribedVerificationNumber();
final CreateVerificationSessionRequest input = new CreateVerificationSessionRequest(params.number(),
originalRequest);
final VerificationSessionResponse verificationSessionResponse = Operations
.apiPost("/v1/verification/session", input)
.executeExpectSuccess(VerificationSessionResponse.class);
final String sessionId = verificationSessionResponse.id();
Assertions.assertTrue(StringUtils.isNotBlank(sessionId));
final String pushChallenge = Operations.peekVerificationSessionPushChallenge(sessionId);
// supply push challenge
@@ -46,7 +52,8 @@ public class RegistrationTest {
.executeExpectSuccess(VerificationSessionResponse.class);
// verify code
final SubmitVerificationCodeRequest submitVerificationCodeRequest = new SubmitVerificationCodeRequest("265402");
final SubmitVerificationCodeRequest submitVerificationCodeRequest = new SubmitVerificationCodeRequest(
params.verificationCode());
final VerificationSessionResponse codeVerified = Operations
.apiPut("/v1/verification/session/%s/code".formatted(sessionId), submitVerificationCodeRequest)
.executeExpectSuccess(VerificationSessionResponse.class);

218
mvnw vendored
View File

@@ -19,7 +19,7 @@
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
# Maven Start Up Batch script
# Apache Maven Wrapper startup batch script, version 3.2.0
#
# Required ENV vars:
# ------------------
@@ -27,7 +27,6 @@
#
# Optional ENV vars
# -----------------
# M2_HOME - location of maven2's installed home dir
# MAVEN_OPTS - parameters passed to the Java VM when running Maven
# e.g. to debug Maven itself, use
# set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
@@ -54,7 +53,7 @@ fi
cygwin=false;
darwin=false;
mingw=false
case "`uname`" in
case "$(uname)" in
CYGWIN*) cygwin=true ;;
MINGW*) mingw=true;;
Darwin*) darwin=true
@@ -62,9 +61,9 @@ case "`uname`" in
# See https://developer.apple.com/library/mac/qa/qa1170/_index.html
if [ -z "$JAVA_HOME" ]; then
if [ -x "/usr/libexec/java_home" ]; then
export JAVA_HOME="`/usr/libexec/java_home`"
JAVA_HOME="$(/usr/libexec/java_home)"; export JAVA_HOME
else
export JAVA_HOME="/Library/Java/Home"
JAVA_HOME="/Library/Java/Home"; export JAVA_HOME
fi
fi
;;
@@ -72,68 +71,38 @@ esac
if [ -z "$JAVA_HOME" ] ; then
if [ -r /etc/gentoo-release ] ; then
JAVA_HOME=`java-config --jre-home`
JAVA_HOME=$(java-config --jre-home)
fi
fi
if [ -z "$M2_HOME" ] ; then
## resolve links - $0 may be a link to maven's home
PRG="$0"
# need this for relative symlinks
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG="`dirname "$PRG"`/$link"
fi
done
saveddir=`pwd`
M2_HOME=`dirname "$PRG"`/..
# make it fully qualified
M2_HOME=`cd "$M2_HOME" && pwd`
cd "$saveddir"
# echo Using m2 at $M2_HOME
fi
# For Cygwin, ensure paths are in UNIX format before anything is touched
if $cygwin ; then
[ -n "$M2_HOME" ] &&
M2_HOME=`cygpath --unix "$M2_HOME"`
[ -n "$JAVA_HOME" ] &&
JAVA_HOME=`cygpath --unix "$JAVA_HOME"`
JAVA_HOME=$(cygpath --unix "$JAVA_HOME")
[ -n "$CLASSPATH" ] &&
CLASSPATH=`cygpath --path --unix "$CLASSPATH"`
CLASSPATH=$(cygpath --path --unix "$CLASSPATH")
fi
# For Mingw, ensure paths are in UNIX format before anything is touched
if $mingw ; then
[ -n "$M2_HOME" ] &&
M2_HOME="`(cd "$M2_HOME"; pwd)`"
[ -n "$JAVA_HOME" ] &&
JAVA_HOME="`(cd "$JAVA_HOME"; pwd)`"
[ -n "$JAVA_HOME" ] && [ -d "$JAVA_HOME" ] &&
JAVA_HOME="$(cd "$JAVA_HOME" || (echo "cannot cd into $JAVA_HOME."; exit 1); pwd)"
fi
if [ -z "$JAVA_HOME" ]; then
javaExecutable="`which javac`"
if [ -n "$javaExecutable" ] && ! [ "`expr \"$javaExecutable\" : '\([^ ]*\)'`" = "no" ]; then
javaExecutable="$(which javac)"
if [ -n "$javaExecutable" ] && ! [ "$(expr "\"$javaExecutable\"" : '\([^ ]*\)')" = "no" ]; then
# readlink(1) is not available as standard on Solaris 10.
readLink=`which readlink`
if [ ! `expr "$readLink" : '\([^ ]*\)'` = "no" ]; then
readLink=$(which readlink)
if [ ! "$(expr "$readLink" : '\([^ ]*\)')" = "no" ]; then
if $darwin ; then
javaHome="`dirname \"$javaExecutable\"`"
javaExecutable="`cd \"$javaHome\" && pwd -P`/javac"
javaHome="$(dirname "\"$javaExecutable\"")"
javaExecutable="$(cd "\"$javaHome\"" && pwd -P)/javac"
else
javaExecutable="`readlink -f \"$javaExecutable\"`"
javaExecutable="$(readlink -f "\"$javaExecutable\"")"
fi
javaHome="`dirname \"$javaExecutable\"`"
javaHome=`expr "$javaHome" : '\(.*\)/bin'`
javaHome="$(dirname "\"$javaExecutable\"")"
javaHome=$(expr "$javaHome" : '\(.*\)/bin')
JAVA_HOME="$javaHome"
export JAVA_HOME
fi
@@ -149,7 +118,7 @@ if [ -z "$JAVACMD" ] ; then
JAVACMD="$JAVA_HOME/bin/java"
fi
else
JAVACMD="`\\unset -f command; \\command -v java`"
JAVACMD="$(\unset -f command 2>/dev/null; \command -v java)"
fi
fi
@@ -163,12 +132,9 @@ if [ -z "$JAVA_HOME" ] ; then
echo "Warning: JAVA_HOME environment variable is not set."
fi
CLASSWORLDS_LAUNCHER=org.codehaus.plexus.classworlds.launcher.Launcher
# traverses directory structure from process work directory to filesystem root
# first directory with .mvn subdirectory is considered project base directory
find_maven_basedir() {
if [ -z "$1" ]
then
echo "Path not specified to find_maven_basedir"
@@ -184,96 +150,99 @@ find_maven_basedir() {
fi
# workaround for JBEAP-8937 (on Solaris 10/Sparc)
if [ -d "${wdir}" ]; then
wdir=`cd "$wdir/.."; pwd`
wdir=$(cd "$wdir/.." || exit 1; pwd)
fi
# end of workaround
done
echo "${basedir}"
printf '%s' "$(cd "$basedir" || exit 1; pwd)"
}
# concatenates all lines of a file
concat_lines() {
if [ -f "$1" ]; then
echo "$(tr -s '\n' ' ' < "$1")"
# Remove \r in case we run on Windows within Git Bash
# and check out the repository with auto CRLF management
# enabled. Otherwise, we may read lines that are delimited with
# \r\n and produce $'-Xarg\r' rather than -Xarg due to word
# splitting rules.
tr -s '\r\n' ' ' < "$1"
fi
}
BASE_DIR=`find_maven_basedir "$(pwd)"`
log() {
if [ "$MVNW_VERBOSE" = true ]; then
printf '%s\n' "$1"
fi
}
BASE_DIR=$(find_maven_basedir "$(dirname "$0")")
if [ -z "$BASE_DIR" ]; then
exit 1;
fi
MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-"$BASE_DIR"}; export MAVEN_PROJECTBASEDIR
log "$MAVEN_PROJECTBASEDIR"
##########################################################################################
# Extension to allow automatically downloading the maven-wrapper.jar from Maven-central
# This allows using the maven wrapper in projects that prohibit checking in binary data.
##########################################################################################
if [ -r "$BASE_DIR/.mvn/wrapper/maven-wrapper.jar" ]; then
if [ "$MVNW_VERBOSE" = true ]; then
echo "Found .mvn/wrapper/maven-wrapper.jar"
fi
wrapperJarPath="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar"
if [ -r "$wrapperJarPath" ]; then
log "Found $wrapperJarPath"
else
if [ "$MVNW_VERBOSE" = true ]; then
echo "Couldn't find .mvn/wrapper/maven-wrapper.jar, downloading it ..."
fi
log "Couldn't find $wrapperJarPath, downloading it ..."
if [ -n "$MVNW_REPOURL" ]; then
jarUrl="$MVNW_REPOURL/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
wrapperUrl="$MVNW_REPOURL/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
else
jarUrl="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
wrapperUrl="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
fi
while IFS="=" read key value; do
case "$key" in (wrapperUrl) jarUrl="$value"; break ;;
while IFS="=" read -r key value; do
# Remove '\r' from value to allow usage on windows as IFS does not consider '\r' as a separator ( considers space, tab, new line ('\n'), and custom '=' )
safeValue=$(echo "$value" | tr -d '\r')
case "$key" in (wrapperUrl) wrapperUrl="$safeValue"; break ;;
esac
done < "$BASE_DIR/.mvn/wrapper/maven-wrapper.properties"
if [ "$MVNW_VERBOSE" = true ]; then
echo "Downloading from: $jarUrl"
fi
wrapperJarPath="$BASE_DIR/.mvn/wrapper/maven-wrapper.jar"
done < "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.properties"
log "Downloading from: $wrapperUrl"
if $cygwin; then
wrapperJarPath=`cygpath --path --windows "$wrapperJarPath"`
wrapperJarPath=$(cygpath --path --windows "$wrapperJarPath")
fi
if command -v wget > /dev/null; then
if [ "$MVNW_VERBOSE" = true ]; then
echo "Found wget ... using wget"
fi
log "Found wget ... using wget"
[ "$MVNW_VERBOSE" = true ] && QUIET="" || QUIET="--quiet"
if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then
wget "$jarUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
wget $QUIET "$wrapperUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
else
wget --http-user=$MVNW_USERNAME --http-password=$MVNW_PASSWORD "$jarUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
wget $QUIET --http-user="$MVNW_USERNAME" --http-password="$MVNW_PASSWORD" "$wrapperUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
fi
elif command -v curl > /dev/null; then
if [ "$MVNW_VERBOSE" = true ]; then
echo "Found curl ... using curl"
fi
log "Found curl ... using curl"
[ "$MVNW_VERBOSE" = true ] && QUIET="" || QUIET="--silent"
if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then
curl -o "$wrapperJarPath" "$jarUrl" -f
curl $QUIET -o "$wrapperJarPath" "$wrapperUrl" -f -L || rm -f "$wrapperJarPath"
else
curl --user $MVNW_USERNAME:$MVNW_PASSWORD -o "$wrapperJarPath" "$jarUrl" -f
curl $QUIET --user "$MVNW_USERNAME:$MVNW_PASSWORD" -o "$wrapperJarPath" "$wrapperUrl" -f -L || rm -f "$wrapperJarPath"
fi
else
if [ "$MVNW_VERBOSE" = true ]; then
echo "Falling back to using Java to download"
fi
javaClass="$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.java"
log "Falling back to using Java to download"
javaSource="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/MavenWrapperDownloader.java"
javaClass="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/MavenWrapperDownloader.class"
# For Cygwin, switch paths to Windows format before running javac
if $cygwin; then
javaClass=`cygpath --path --windows "$javaClass"`
javaSource=$(cygpath --path --windows "$javaSource")
javaClass=$(cygpath --path --windows "$javaClass")
fi
if [ -e "$javaClass" ]; then
if [ ! -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then
if [ "$MVNW_VERBOSE" = true ]; then
echo " - Compiling MavenWrapperDownloader.java ..."
fi
# Compiling the Java class
("$JAVA_HOME/bin/javac" "$javaClass")
if [ -e "$javaSource" ]; then
if [ ! -e "$javaClass" ]; then
log " - Compiling MavenWrapperDownloader.java ..."
("$JAVA_HOME/bin/javac" "$javaSource")
fi
if [ -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then
# Running the downloader
if [ "$MVNW_VERBOSE" = true ]; then
echo " - Running MavenWrapperDownloader.java ..."
fi
("$JAVA_HOME/bin/java" -cp .mvn/wrapper MavenWrapperDownloader "$MAVEN_PROJECTBASEDIR")
if [ -e "$javaClass" ]; then
log " - Running MavenWrapperDownloader.java ..."
("$JAVA_HOME/bin/java" -cp .mvn/wrapper MavenWrapperDownloader "$wrapperUrl" "$wrapperJarPath") || rm -f "$wrapperJarPath"
fi
fi
fi
@@ -282,35 +251,58 @@ fi
# End of extension
##########################################################################################
export MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-"$BASE_DIR"}
if [ "$MVNW_VERBOSE" = true ]; then
echo $MAVEN_PROJECTBASEDIR
# If specified, validate the SHA-256 sum of the Maven wrapper jar file
wrapperSha256Sum=""
while IFS="=" read -r key value; do
case "$key" in (wrapperSha256Sum) wrapperSha256Sum=$value; break ;;
esac
done < "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.properties"
if [ -n "$wrapperSha256Sum" ]; then
wrapperSha256Result=false
if command -v sha256sum > /dev/null; then
if echo "$wrapperSha256Sum $wrapperJarPath" | sha256sum -c - > /dev/null 2>&1; then
wrapperSha256Result=true
fi
elif command -v shasum > /dev/null; then
if echo "$wrapperSha256Sum $wrapperJarPath" | shasum -a 256 -c > /dev/null 2>&1; then
wrapperSha256Result=true
fi
else
echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available."
echo "Please install either command, or disable validation by removing 'wrapperSha256Sum' from your maven-wrapper.properties."
exit 1
fi
if [ $wrapperSha256Result = false ]; then
echo "Error: Failed to validate Maven wrapper SHA-256, your Maven wrapper might be compromised." >&2
echo "Investigate or delete $wrapperJarPath to attempt a clean download." >&2
echo "If you updated your Maven version, you need to update the specified wrapperSha256Sum property." >&2
exit 1
fi
fi
MAVEN_OPTS="$(concat_lines "$MAVEN_PROJECTBASEDIR/.mvn/jvm.config") $MAVEN_OPTS"
# For Cygwin, switch paths to Windows format before running java
if $cygwin; then
[ -n "$M2_HOME" ] &&
M2_HOME=`cygpath --path --windows "$M2_HOME"`
[ -n "$JAVA_HOME" ] &&
JAVA_HOME=`cygpath --path --windows "$JAVA_HOME"`
JAVA_HOME=$(cygpath --path --windows "$JAVA_HOME")
[ -n "$CLASSPATH" ] &&
CLASSPATH=`cygpath --path --windows "$CLASSPATH"`
CLASSPATH=$(cygpath --path --windows "$CLASSPATH")
[ -n "$MAVEN_PROJECTBASEDIR" ] &&
MAVEN_PROJECTBASEDIR=`cygpath --path --windows "$MAVEN_PROJECTBASEDIR"`
MAVEN_PROJECTBASEDIR=$(cygpath --path --windows "$MAVEN_PROJECTBASEDIR")
fi
# Provide a "standardized" way to retrieve the CLI args that will
# work with both Windows and non-Windows executions.
MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $@"
MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $*"
export MAVEN_CMD_LINE_ARGS
WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain
# shellcheck disable=SC2086 # safe args
exec "$JAVACMD" \
$MAVEN_OPTS \
$MAVEN_DEBUG_OPTS \
-classpath "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar" \
"-Dmaven.home=${M2_HOME}" \
"-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \
${WRAPPER_LAUNCHER} $MAVEN_CONFIG "$@"

31
mvnw.cmd vendored
View File

@@ -18,13 +18,12 @@
@REM ----------------------------------------------------------------------------
@REM ----------------------------------------------------------------------------
@REM Maven Start Up Batch script
@REM Apache Maven Wrapper startup batch script, version 3.2.0
@REM
@REM Required ENV vars:
@REM JAVA_HOME - location of a JDK home dir
@REM
@REM Optional ENV vars
@REM M2_HOME - location of maven2's installed home dir
@REM MAVEN_BATCH_ECHO - set to 'on' to enable the echoing of the batch commands
@REM MAVEN_BATCH_PAUSE - set to 'on' to wait for a keystroke before ending
@REM MAVEN_OPTS - parameters passed to the Java VM when running Maven
@@ -120,10 +119,10 @@ SET MAVEN_JAVA_EXE="%JAVA_HOME%\bin\java.exe"
set WRAPPER_JAR="%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.jar"
set WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain
set DOWNLOAD_URL="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
set WRAPPER_URL="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
FOR /F "usebackq tokens=1,2 delims==" %%A IN ("%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.properties") DO (
IF "%%A"=="wrapperUrl" SET DOWNLOAD_URL=%%B
IF "%%A"=="wrapperUrl" SET WRAPPER_URL=%%B
)
@REM Extension to allow automatically downloading the maven-wrapper.jar from Maven-central
@@ -134,11 +133,11 @@ if exist %WRAPPER_JAR% (
)
) else (
if not "%MVNW_REPOURL%" == "" (
SET DOWNLOAD_URL="%MVNW_REPOURL%/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
SET WRAPPER_URL="%MVNW_REPOURL%/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
)
if "%MVNW_VERBOSE%" == "true" (
echo Couldn't find %WRAPPER_JAR%, downloading it ...
echo Downloading from: %DOWNLOAD_URL%
echo Downloading from: %WRAPPER_URL%
)
powershell -Command "&{"^
@@ -146,7 +145,7 @@ if exist %WRAPPER_JAR% (
"if (-not ([string]::IsNullOrEmpty('%MVNW_USERNAME%') -and [string]::IsNullOrEmpty('%MVNW_PASSWORD%'))) {"^
"$webclient.Credentials = new-object System.Net.NetworkCredential('%MVNW_USERNAME%', '%MVNW_PASSWORD%');"^
"}"^
"[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $webclient.DownloadFile('%DOWNLOAD_URL%', '%WRAPPER_JAR%')"^
"[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $webclient.DownloadFile('%WRAPPER_URL%', '%WRAPPER_JAR%')"^
"}"
if "%MVNW_VERBOSE%" == "true" (
echo Finished downloading %WRAPPER_JAR%
@@ -154,6 +153,24 @@ if exist %WRAPPER_JAR% (
)
@REM End of extension
@REM If specified, validate the SHA-256 sum of the Maven wrapper jar file
SET WRAPPER_SHA_256_SUM=""
FOR /F "usebackq tokens=1,2 delims==" %%A IN ("%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.properties") DO (
IF "%%A"=="wrapperSha256Sum" SET WRAPPER_SHA_256_SUM=%%B
)
IF NOT %WRAPPER_SHA_256_SUM%=="" (
powershell -Command "&{"^
"$hash = (Get-FileHash \"%WRAPPER_JAR%\" -Algorithm SHA256).Hash.ToLower();"^
"If('%WRAPPER_SHA_256_SUM%' -ne $hash){"^
" Write-Output 'Error: Failed to validate Maven wrapper SHA-256, your Maven wrapper might be compromised.';"^
" Write-Output 'Investigate or delete %WRAPPER_JAR% to attempt a clean download.';"^
" Write-Output 'If you updated your Maven version, you need to update the specified wrapperSha256Sum property.';"^
" exit 1;"^
"}"^
"}"
if ERRORLEVEL 1 goto error
)
@REM Provide a "standardized" way to retrieve the CLI args that will
@REM work with both Windows and non-Windows executions.
set MAVEN_CMD_LINE_ARGS=%*

318
pom.xml
View File

@@ -31,45 +31,66 @@
<modules>
<module>api-doc</module>
<module>event-logger</module>
<module>integration-tests</module>
<module>service</module>
<module>websocket-resources</module>
</modules>
<properties>
<aws.sdk2.version>2.20.118</aws.sdk2.version>
<braintree.version>3.19.0</braintree.version>
<commons-csv.version>1.9.0</commons-csv.version>
<commons-io.version>2.9.0</commons-io.version>
<dropwizard.version>2.1.7</dropwizard.version>
<dropwizard-metrics-datadog.version>1.1.13</dropwizard-metrics-datadog.version>
<google-cloud-libraries.version>26.21.0</google-cloud-libraries.version>
<grpc.version>1.56.1</grpc.version> <!-- should be kept in sync with the value from Google libraries-bom -->
<gson.version>2.9.0</gson.version>
<jackson.version>2.13.5</jackson.version>
<jaxb.version>2.3.1</jaxb.version>
<kotlin.version>1.8.0</kotlin.version>
<kotlinx-serialization.version>1.4.1</kotlinx-serialization.version>
<lettuce.version>6.2.4.RELEASE</lettuce.version>
<libphonenumber.version>8.12.54</libphonenumber.version>
<logstash.logback.version>7.2</logstash.logback.version>
<luajava.version>3.4.0</luajava.version>
<micrometer.version>1.10.3</micrometer.version>
<mockito.version>4.11.0</mockito.version>
<netty.version>4.1.96.Final</netty.version>
<opentest4j.version>1.2.0</opentest4j.version>
<protobuf.version>3.23.2</protobuf.version> <!-- should be kept in sync with the value from Google libraries-bom -->
<pushy.version>0.15.2</pushy.version>
<aws.sdk2.version>2.33.8</aws.sdk2.version>
<braintree.version>3.44.0</braintree.version>
<commons-csv.version>1.14.1</commons-csv.version>
<commons-io.version>2.20.0</commons-io.version>
<dropwizard.version>4.0.16</dropwizard.version>
<!-- Note: when updating FoundationDB, also include a copy of `libfdb_c.so` from the FoundationDB release at
src/main/jib/usr/lib/libfdb_c.so. We use x86_64 builds without AVX instructions enabled (i.e. FoundationDB versions
with even-numbered patch versions). Also when updating FoundationDB, make sure to update the version of FoundationDB
used by GitHub Actions. -->
<foundationdb.version>7.3.62</foundationdb.version>
<foundationdb.api-version>730</foundationdb.api-version>
<foundationdb.client-library-sha256>bfed237b787fae3cde1222676e6bfbb0d218fc27bf9e903397a7a7aa96fb2d33</foundationdb.client-library-sha256>
<google-cloud-libraries.version>26.67.0</google-cloud-libraries.version>
<grpc.version>1.73.0</grpc.version> <!-- should be kept in sync with the value from Google libraries-bom -->
<gson.version>2.13.2</gson.version>
<!-- several libraries (AWS, Google Cloud) use Apache http components transitively, and we need to align them -->
<httpcore.version>4.4.16</httpcore.version>
<httpclient.version>4.5.14</httpclient.version>
<jackson.version>2.20.0</jackson.version>
<junit-pioneer.version>2.3.0</junit-pioneer.version>
<jsr305.version>3.0.2</jsr305.version>
<kotlin.version>2.2.20</kotlin.version>
<!-- Logback 1.5.14+ has a null pointer bug: https://github.com/qos-ch/logback/issues/929. -->
<logback.version>1.5.13</logback.version>
<logback-access-common.version>2.0.6</logback-access-common.version>
<lettuce.version>6.8.1.RELEASE</lettuce.version>
<libphonenumber.version>9.0.13</libphonenumber.version>
<logstash.logback.version>8.1</logstash.logback.version>
<log4j-bom.version>2.25.1</log4j-bom.version>
<luajava.version>3.5.0</luajava.version>
<micrometer.version>1.15.4</micrometer.version>
<netty.version>4.1.127.Final</netty.version>
<!-- Must be less than or equal to the value from Google libraries-bom which controls the protobuf runtime version.
See https://protobuf.dev/support/cross-version-runtime-guarantee/. -->
<protoc.version>4.29.4</protoc.version>
<pushy.version>0.15.4</pushy.version>
<reactive.grpc.version>1.2.4</reactive.grpc.version>
<resilience4j.version>1.7.0</resilience4j.version>
<reactor-bom.version>2024.0.10</reactor-bom.version> <!-- 3.7.11, see https://github.com/reactor/reactor#bom-versioning-scheme -->
<resilience4j.version>2.3.0</resilience4j.version>
<semver4j.version>3.1.0</semver4j.version>
<slf4j.version>1.7.30</slf4j.version>
<stripe.version>21.2.0</stripe.version>
<vavr.version>0.10.4</vavr.version>
<simple-grpc.version>0.1.0</simple-grpc.version>
<slf4j.version>2.0.17</slf4j.version>
<stripe.version>30.2.0</stripe.version>
<swagger.version>2.2.36</swagger.version>
<testcontainers.version>1.21.3</testcontainers.version>
<!-- 17.0.7_7-jre-jammy -->
<docker.image.sha256>ddf36656dc8920621fddf4928bdcb4b98c0d0e7bc9672f0cea8115c10ad5cbc6</docker.image.sha256>
<!-- images to use in tests via testcontainers -->
<dynamodb.image>amazon/dynamodb-local:3.0.0@sha256:2fed5e3a965a4ba5aa6ac82baec57058b5a3848e959d705518f3fd579a77e76b</dynamodb.image>
<localstack.image>localstack/localstack:4@sha256:5a97e0f9917a3f0d9630bb13b9d8ccf10cbe52f33252807d3b4e21418cc21348</localstack.image>
<redis.image>redis:7.4-alpine@sha256:af1d0fc3f63b02b13ff7906c9baf7c5b390b8881ca08119cd570677fe2f60b55</redis.image>
<redis-cluster.image>docker.io/bitnamilegacy/redis-cluster:7.4.3@sha256:a53d023fdfaf8a8d7ddc58da040d3494e4cb45772644618ffa44c42dcd32b9af</redis-cluster.image>
<!-- eclipse-temurin:24.0.2_12-jre-noble (note: always use the multi-arch manifest *LIST* here) -->
<docker.image.sha256>85ecfc9bbb42af046d2bacbf1219d2005be4840cbfa16c2e6fd910d9ccfec95b</docker.image.sha256>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
@@ -141,10 +162,24 @@
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-bom</artifactId>
<version>1.54.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-instrumentation-bom</artifactId>
<version>2.20.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-bom</artifactId>
<version>2022.0.3</version> <!-- 3.5.x, see https://github.com/reactor/reactor#bom-versioning-scheme -->
<version>${reactor-bom.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
@@ -165,11 +200,6 @@
<artifactId>pushy-dropwizard-metrics-listener</artifactId>
<version>${pushy.version}</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>${protobuf.version}</version>
</dependency>
<dependency>
<groupId>com.googlecode.libphonenumber</groupId>
<artifactId>libphonenumber</artifactId>
@@ -190,16 +220,6 @@
<artifactId>lettuce-core</artifactId>
<version>${lettuce.version}</version>
</dependency>
<dependency>
<groupId>io.vavr</groupId>
<artifactId>vavr</artifactId>
<version>${vavr.version}</version>
</dependency>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>${jaxb.version}</version>
</dependency>
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
@@ -211,33 +231,9 @@
<version>${commons-csv.version}</version>
</dependency>
<dependency>
<groupId>org.coursera</groupId>
<artifactId>dropwizard-metrics-datadog</artifactId>
<version>${dropwizard-metrics-datadog.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
<version>${jaxb.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.opentest4j</groupId>
<artifactId>opentest4j</artifactId>
<version>${opentest4j.version}</version>
<scope>test</scope>
<groupId>org.foundationdb</groupId>
<artifactId>fdb-java</artifactId>
<version>${foundationdb.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
@@ -253,12 +249,12 @@
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
<version>1.3.5</version>
</dependency>
<dependency>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
<version>9.2</version>
<version>9.8</version>
<scope>test</scope>
</dependency>
<dependency>
@@ -271,29 +267,77 @@
<artifactId>braintree-java</artifactId>
<version>${braintree.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>${jsr305.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
</dependency>
<dependency>
<groupId>org.signal</groupId>
<artifactId>embedded-redis</artifactId>
<version>0.8.3</version>
<groupId>com.redis</groupId>
<artifactId>testcontainers-redis</artifactId>
<version>2.2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.signal</groupId>
<artifactId>libsignal-server</artifactId>
<version>0.30.0</version>
<version>0.80.3</version>
</dependency>
<dependency>
<groupId>org.signal</groupId>
<artifactId>simple-grpc-runtime</artifactId>
<version>${simple-grpc.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-bom</artifactId>
<version>2.17.1</version>
<version>${log4j-bom.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>${httpcore.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${httpclient.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback.access</groupId>
<artifactId>logback-access-common</artifactId>
<version>${logback-access-common.version}</version>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-bom</artifactId>
<version>${testcontainers.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>earth.adi</groupId>
<artifactId>testcontainers-foundationdb</artifactId>
<version>1.1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</dependencyManagement>
@@ -305,25 +349,18 @@
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.github.tomakehurst</groupId>
<artifactId>wiremock-jre8</artifactId>
<version>2.35.0</version>
<groupId>software.amazon.awssdk</groupId>
<artifactId>aws-crt-client</artifactId>
</dependency>
<dependency>
<groupId>org.wiremock</groupId>
<artifactId>wiremock</artifactId>
<version>3.13.1</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-core</artifactId>
</exclusion>
<exclusion>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
@@ -339,7 +376,7 @@
<dependency>
<groupId>org.junit-pioneer</groupId>
<artifactId>junit-pioneer</artifactId>
<version>1.9.1</version>
<version>${junit-pioneer.version}</version>
<scope>test</scope>
</dependency>
@@ -381,7 +418,42 @@
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>3.3.1</version>
<version>3.4.4</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.5.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.4.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.6.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.2.1</version>
</plugin>
</plugins>
</pluginManagement>
@@ -393,7 +465,7 @@
<version>0.6.1</version>
<configuration>
<checkStaleness>false</checkStaleness>
<protocArtifact>com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}</protocArtifact>
<protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
@@ -405,6 +477,14 @@
<version>${reactive.grpc.version}</version>
<mainClass>com.salesforce.reactorgrpc.ReactorGrpcGenerator</mainClass>
</protocPlugin>
<protocPlugin>
<id>simple</id>
<groupId>org.signal</groupId>
<artifactId>simple-grpc-generator</artifactId>
<version>${simple-grpc.version}</version>
<mainClass>org.signal.grpc.simple.SimpleGrpcGenerator</mainClass>
</protocPlugin>
</protocPlugins>
</configuration>
<executions>
@@ -422,16 +502,16 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<version>3.13.0</version>
<configuration>
<release>17</release>
<release>24</release>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.2.0</version>
<version>3.4.2</version>
<configuration>
<archive>
<manifest>
@@ -444,41 +524,27 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.2</version>
<version>3.8.1</version>
<executions>
<execution>
<id>copy</id>
<phase>test-compile</phase>
<!--
Set dependencies as properties for use in argLine property for mockito jar.
The property isn't needed until the test phase, and deferring it from the default
`initialize` addresses issues running lifecycle phases that precede `test` in isolation.
-->
<phase>process-test-classes</phase>
<goals>
<goal>copy-dependencies</goal>
<goal>properties</goal>
</goals>
<configuration>
<includeScope>test</includeScope>
<includeTypes>so,dll,dylib</includeTypes>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<systemProperties>
<property>
<name>sqlite4java.library.path</name>
<value>${project.build.directory}/lib</value>
</property>
</systemProperties>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.0.0-M3</version>
<version>3.5.0</version>
<executions>
<execution>
<goals>
@@ -488,7 +554,7 @@
<rules>
<dependencyConvergence/>
<requireMavenVersion>
<version>3.8.6</version>
<version>3.9.11</version>
</requireMavenVersion>
</rules>
</configuration>
@@ -499,7 +565,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>3.0.0-M1</version>
<version>3.1.3</version>
<configuration>
<skip>true</skip>
</configuration>
@@ -508,7 +574,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>3.0.0-M1</version>
<version>3.1.3</version>
<configuration>
<skip>true</skip>
</configuration>

View File

@@ -1,20 +1,24 @@
datadog.apiKey: unset
stripe.apiKey: unset
stripe.idempotencyKeyGenerator: abcdefg12345678= # base64 for creating request idempotency hash
braintree.publicKey: unset
braintree.privateKey: unset
googlePlayBilling.credentialsJson: |
{ "json": true }
appleAppStore.encodedKey: unset
directoryV2.client.userAuthenticationTokenSharedSecret: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with CDS to generate auth tokens for Signal users
directoryV2.client.userIdTokenSharedSecret: bbcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with CDS to generate auth identity tokens for Signal users
svr2.userAuthenticationTokenSharedSecret: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with SVR2 to generate auth tokens for Signal users
svr2.userIdTokenSharedSecret: bbcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with SVR2 to generate auth identity tokens for Signal users
tus.userAuthenticationTokenSharedSecret: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFG=
svrb.userAuthenticationTokenSharedSecret: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with SVRB to generate auth tokens for Signal users
svrb.userIdTokenSharedSecret: bbcdefghijklmnopqrstuvwxyz0123456789ABCDEFG= # base64-encoded secret shared with SVRB to generate auth identity tokens for Signal users
awsAttachments.accessKey: test
awsAttachments.accessSecret: test
tus.userAuthenticationTokenSharedSecret: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFG=
gcpAttachments.rsaSigningKey: |
-----BEGIN PRIVATE KEY-----
@@ -46,6 +50,8 @@ gcpAttachments.rsaSigningKey: |
AAAAAAAA
-----END PRIVATE KEY-----
apn.teamId: team-id
apn.keyId: key-id
apn.signingKey: |
-----BEGIN PRIVATE KEY-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
@@ -60,29 +66,37 @@ fcm.credentials: |
cdn.accessKey: test # AWS Access Key ID
cdn.accessSecret: test # AWS Access Secret
unidentifiedDelivery.certificate: ABCD1234
cdn3StorageManager.clientSecret: test
unidentifiedDelivery.privateKey: ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789AAAAAAA
hCaptcha.apiKey: unset
keyTransparencyService.clientPrivateKey: |
-----BEGIN PRIVATE KEY-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
AAAAAAAA
-----END PRIVATE KEY-----
storageService.userAuthenticationTokenSharedSecret: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
backupService.userAuthenticationTokenSharedSecret: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
zkConfig.serverSecret: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzAA==
zkConfig-libsignal-0.42.serverSecret: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdef
genericZkConfig.serverSecret: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzAA==
callingZkConfig.serverSecret: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzAA==
backupsZkConfig.serverSecret: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzAA==
paymentsService.userAuthenticationTokenSharedSecret: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= # base64-encoded 32-byte secret shared with MobileCoin services used to generate auth tokens for Signal users
paymentsService.fixerApiKey: unset
paymentsService.coinMarketCapApiKey: unset
artService.userAuthenticationTokenSharedSecret: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= # base64-encoded 32-byte secret not shared with any external service, but used in ArtController
artService.userAuthenticationTokenUserIdSecret: AAAAAAAAAAA= # base64-encoded secret to obscure user phone numbers from Sticker Creator
paymentsService.coinGeckoApiKey: unset
currentReportingKey.secret: AAAAAAAAAAA=
currentReportingKey.salt: AAAAAAAAAAA=
turn.secret: AAAAAAAAAAA=
registrationService.collationKeySalt: AAAAAAAAAAA=
turn.cloudflare.apiToken: ABCDEFGHIJKLM
linkDevice.secret: AAAAAAAAAAA=
tlsKeyStore.password: unset

View File

@@ -10,72 +10,70 @@ logging:
threshold: ALL
timeZone: UTC
target: stdout
- type: logstashtcpsocket
destination: example.com:10516
apiKey: secret://datadog.apiKey
environment: staging
- type: otlp
metrics:
reporters:
- type: signal-datadog
frequency: 10 seconds
tags:
- "env:staging"
- "service:chat"
udpTransport:
statsdHost: localhost
port: 8125
excludesAttributes:
- m1_rate
- m5_rate
- m15_rate
- mean_rate
- stddev
useRegexFilters: true
excludes:
- ^.+\.total$
- ^.+\.request\.filtering$
- ^.+\.response\.filtering$
- ^executor\..+$
- ^lettuce\..+$
reportOnStop: true
adminEventLoggingConfiguration:
credentials: |
{
"key": "value"
}
projectId: some-project-id
logName: some-log-name
grpcPort: 8080
tlsKeyStore:
password: secret://tlsKeyStore.password
stripe:
apiKey: secret://stripe.apiKey
idempotencyKeyGenerator: secret://stripe.idempotencyKeyGenerator
boostDescription: >
Example
supportedCurrencies:
- xts
# - ...
# - Nth supported currency
supportedCurrenciesByPaymentMethod:
CARD:
- usd
- eur
SEPA_DEBIT:
- eur
braintree:
merchantId: unset
publicKey: unset
publicKey: secret://braintree.publicKey
privateKey: secret://braintree.privateKey
environment: unset
graphqlUrl: unset
merchantAccounts:
# ISO 4217 currency code and its corresponding sub-merchant account
'xts': unset
supportedCurrencies:
- xts
# - ...
# - Nth supported currency
supportedCurrenciesByPaymentMethod:
PAYPAL:
- usd
pubSubPublisher:
project: example-project
topic: example-topic
credentialConfiguration: |
{
"credential": "configuration"
}
dynamoDbClientConfiguration:
googlePlayBilling:
credentialsJson: secret://googlePlayBilling.credentialsJson
packageName: package.name
applicationName: test
productIdToLevel: {}
appleAppStore:
env: SANDBOX
bundleId: bundle.name
appAppleId: 12345
issuerId: abcdefg
keyId: abcdefg
encodedKey: secret://appleAppStore.encodedKey
subscriptionGroupId: example_subscriptionGroupId
productIdToLevel: {}
appleRootCerts: []
appleDeviceCheck:
production: false
teamId: 0123456789
bundleId: bundle.name
deviceCheck:
backupRedemptionDuration: P30D
backupRedemptionLevel: 201
dynamoDbClient:
region: us-west-2 # AWS Region
dynamoDbTables:
@@ -84,7 +82,13 @@ dynamoDbTables:
phoneNumberTableName: Example_Accounts_PhoneNumbers
phoneNumberIdentifierTableName: Example_Accounts_PhoneNumberIdentifiers
usernamesTableName: Example_Accounts_Usernames
scanPageSize: 100
usedLinkDeviceTokensTableName: Example_Accounts_UsedLinkDeviceTokens
appleDeviceChecks:
tableName: Example_AppleDeviceChecks
appleDeviceCheckPublicKeys:
tableName: Example_AppleDeviceCheckPublicKeys
backups:
tableName: Example_Backups
clientReleases:
tableName: Example_ClientReleases
deletedAccounts:
@@ -95,23 +99,35 @@ dynamoDbTables:
tableName: Example_IssuedReceipts
expiration: P30D # Duration of time until rows expire
generator: abcdefg12345678= # random base64-encoded binary sequence
maxIssuedReceiptsPerPaymentId:
STRIPE: 1
BRAINTREE: 1
GOOGLE_PLAY_BILLING: 1
APPLE_APP_STORE: 1
ecKeys:
tableName: Example_Keys
ecSignedPreKeys:
tableName: Example_EC_Signed_Pre_Keys
pqKeys:
tableName: Example_PQ_Keys
pagedPqKeys:
tableName: Example_PQ_Paged_Keys
pqLastResortKeys:
tableName: Example_PQ_Last_Resort_Keys
messages:
tableName: Example_Messages
expiration: P30D # Duration of time until rows expire
onetimeDonations:
tableName: Example_OnetimeDonations
expiration: P90D
phoneNumberIdentifiers:
tableName: Example_PhoneNumberIdentifiers
profiles:
tableName: Example_Profiles
pushChallenge:
tableName: Example_PushChallenge
pushNotificationExperimentSamples:
tableName: Example_PushNotificationExperimentSamples
redeemedReceipts:
tableName: Example_RedeemedReceipts
expiration: P30D # Duration of time until rows expire
@@ -122,15 +138,21 @@ dynamoDbTables:
tableName: Example_RemoteConfig
reportMessage:
tableName: Example_ReportMessage
scheduledJobs:
tableName: Example_ScheduledJobs
expiration: P7D
subscriptions:
tableName: Example_Subscriptions
clientPublicKeys:
tableName: Example_ClientPublicKeys
verificationSessions:
tableName: Example_VerificationSessions
cacheCluster: # Redis server configuration for cache cluster
configurationUri: redis://redis.example.com:6379/
pagedSingleUseKEMPreKeyStore:
bucket: preKeyBucket # S3 Bucket name
region: us-west-2 # AWS region
clientPresenceCluster: # Redis server configuration for client presence cluster
cacheCluster: # Redis server configuration for cache cluster
configurationUri: redis://redis.example.com:6379/
pubsub: # Redis server configuration for pubsub cluster
@@ -175,21 +197,39 @@ svr2:
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
svrb:
uri: svrb.example.com
userAuthenticationTokenSharedSecret: secret://svrb.userAuthenticationTokenSharedSecret
userIdTokenSharedSecret: secret://svrb.userIdTokenSharedSecret
svrCaCertificates:
- |
-----BEGIN CERTIFICATE-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
messageCache: # Redis server configuration for message store cache
persistDelayMinutes: 1
cluster:
configurationUri: redis://redis.example.com:6379/
metricsCluster:
configurationUri: redis://redis.example.com:6379/
awsAttachments: # AWS S3 configuration
accessKey: secret://awsAttachments.accessKey
accessSecret: secret://awsAttachments.accessSecret
bucket: aws-attachments
region: us-west-2
gcpAttachments: # GCP Storage configuration
domain: example.com
email: user@example.cocm
@@ -201,40 +241,41 @@ tus:
uploadUri: https://example.org/upload
userAuthenticationTokenSharedSecret: secret://tus.userAuthenticationTokenSharedSecret
accountDatabaseCrawler:
chunkSize: 10 # accounts per run
apn: # Apple Push Notifications configuration
sandbox: true
bundleId: com.example.textsecuregcm
keyId: unset
teamId: unset
keyId: secret://apn.keyId
teamId: secret://apn.teamId
signingKey: secret://apn.signingKey
fcm: # FCM configuration
credentials: secret://fcm.credentials
cdn:
accessKey: secret://cdn.accessKey
accessSecret: secret://cdn.accessSecret
bucket: cdn # S3 Bucket name
credentials:
accessKeyId: secret://cdn.accessKey
secretAccessKey: secret://cdn.accessSecret
region: us-west-2 # AWS region
dogstatsd:
cdn3StorageManager:
baseUri: https://storage-manager.example.com
clientId: example
clientSecret: secret://cdn3StorageManager.clientSecret
sourceSchemes:
2: gcs
3: r2
openTelemetry:
enabled: true
environment: dev
url: http://127.0.0.1:4318/
unidentifiedDelivery:
certificate: secret://unidentifiedDelivery.certificate
certificate: CgIIAQ==
privateKey: secret://unidentifiedDelivery.privateKey
expiresDays: 7
recaptcha:
projectPath: projects/example
credentialConfigurationJson: "{ }" # service account configuration for backend authentication
hCaptcha:
apiKey: secret://hCaptcha.apiKey
shortCode:
baseUrl: https://example.com/shortcodes/
@@ -265,73 +306,37 @@ storageService:
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
backupService:
uri: backup.example.com
userAuthenticationTokenSharedSecret: secret://backupService.userAuthenticationTokenSharedSecret
backupCaCertificates:
- |
-----BEGIN CERTIFICATE-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
zkConfig:
serverPublic: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
serverSecret: secret://zkConfig.serverSecret
serverPublic: ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyzAB==
serverSecret: secret://zkConfig-libsignal-0.42.serverSecret
genericZkConfig:
serverSecret: secret://genericZkConfig.serverSecret
callingZkConfig:
serverSecret: secret://callingZkConfig.serverSecret
appConfig:
application: example
environment: example
configuration: example
backupsZkConfig:
serverSecret: secret://backupsZkConfig.serverSecret
dynamicConfig:
s3Region: a-region
s3Bucket: a-bucket
objectKey: dynamic-config.yaml
maxSize: 100000
refreshInterval: PT10S
remoteConfig:
authorizedUsers:
- # 1st authorized user
- # 2nd authorized user
- # ...
- # Nth authorized user
requiredHostedDomain: example.com
audiences:
- # 1st audience
- # 2nd audience
- # ...
- # Nth audience
globalConfig: # keys and values that are given to clients on GET /v1/config
EXAMPLE_KEY: VALUE
paymentsService:
userAuthenticationTokenSharedSecret: secret://paymentsService.userAuthenticationTokenSharedSecret
fixerApiKey: secret://paymentsService.fixerApiKey
coinMarketCapApiKey: secret://paymentsService.coinMarketCapApiKey
coinMarketCapCurrencyIds:
MOB: 7878
paymentCurrencies:
# list of symbols for supported currencies
- MOB
artService:
userAuthenticationTokenSharedSecret: secret://artService.userAuthenticationTokenSharedSecret
userAuthenticationTokenUserIdSecret: secret://artService.userAuthenticationTokenUserIdSecret
externalClients:
fixerApiKey: secret://paymentsService.fixerApiKey
coinGeckoApiKey: secret://paymentsService.coinGeckoApiKey
coinGeckoCurrencyIds:
MOB: mobilecoin
badges:
badges:
@@ -354,7 +359,11 @@ badges:
'1': TEST
subscription: # configuration for Stripe subscriptions
badgeExpiration: P30D
badgeGracePeriod: P15D
backupExpiration: P30D
backupGracePeriod: P15D
backupFreeTierMediaDuration: P30D
levels:
500:
badge: EXAMPLE
@@ -367,6 +376,7 @@ subscription: # configuration for Stripe subscriptions
BRAINTREE: plan_example # braintree Plan ID
oneTimeDonations:
sepaMaximumEuros: '10000'
boost:
level: 1
expiration: P90D
@@ -396,6 +406,7 @@ registrationService:
"example": "example"
}
identityTokenAudience: https://registration.example.com
collationKeySalt: secret://registrationService.collationKeySalt
registrationCaCertificate: | # Registration service TLS certificate trust root
-----BEGIN CERTIFICATE-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
@@ -419,11 +430,99 @@ registrationService:
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
turn:
secret: secret://turn.secret
keyTransparencyService:
host: kt.example.com
port: 443
tlsCertificate: |
-----BEGIN CERTIFICATE-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
clientCertificate: |
-----BEGIN CERTIFICATE-----
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ/0123456789+abcdefghijklmnopqrstuvwxyz
AAAAAAAAAAAAAAAAAAAA
-----END CERTIFICATE-----
clientPrivateKey: secret://keyTransparencyService.clientPrivateKey
commandStopListener:
path: /example/path
turn:
cloudflare:
apiToken: secret://turn.cloudflare.apiToken
endpoint: https://rtc.live.cloudflare.com/v1/turn/keys/LMNOP/credentials/generate
urls:
- turn:turn.example.com:80
urlsWithIps:
- turn:%s
- turn:%s:80?transport=tcp
- turns:%s:443?transport=tcp
requestedCredentialTtl: PT24H
clientCredentialTtl: PT12H
hostname: turn.cloudflare.example.com
numHttpClients: 1
linkDevice:
secret: secret://linkDevice.secret
externalRequestFilter:
grpcMethods:
- com.example.grpc.ExampleService/exampleMethod
paths:
- /example
permittedInternalRanges:
- 127.0.0.0/8
idlePrimaryDeviceReminder:
minIdleDuration: P30D
grpc:
port: 50051
asnTable:
s3Region: a-region
s3Bucket: a-bucket
objectKey: asn.tsv
maxSize: 100000
refreshInterval: PT10S
callQualitySurvey:
pubSubPublisher:
project: example-project
topic: example-topic
credentialConfiguration: |
{
"credential": "configuration"
}

View File

@@ -10,13 +10,49 @@
<modelVersion>4.0.0</modelVersion>
<artifactId>service</artifactId>
<properties>
<firebase-admin.version>9.6.0</firebase-admin.version>
<java-uuid-generator.version>5.1.0</java-uuid-generator.version>
<google-androidpublisher.version>v3-rev20250904-2.0.0</google-androidpublisher.version>
<storekit.version>3.6.0</storekit.version>
<webauthn4j.version>0.29.6.RELEASE</webauthn4j.version>
<java-jwt.version>4.5.0</java-jwt.version>
</properties>
<dependencies>
<dependency>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-jaxrs2</artifactId>
<version>2.2.8</version>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>${java-jwt.version}</version>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-androidpublisher</artifactId>
<version>${google-androidpublisher.version}</version>
</dependency>
<dependency>
<groupId>com.apple.itunes.storekit</groupId>
<artifactId>app-store-server-library</artifactId>
<version>${storekit.version}</version>
<exclusions>
<!-- org.yaml:snakeyaml is causing a dependency convergence error -->
<!-- conflicts with other users; resolved manually with explicit import -->
<exclusion>
<groupId>com.squareup.okio</groupId>
<artifactId>okio-jvm</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.webauthn4j</groupId>
<artifactId>webauthn4j-appattest</artifactId>
<version>${webauthn4j.version}</version>
</dependency>
<dependency>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-jaxrs2-jakarta</artifactId>
<version>${swagger.version}</version>
<exclusions>
<!-- conflicts with jackson-dataformat-yaml -->
<exclusion>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
@@ -36,11 +72,6 @@
<artifactId>jakarta.ws.rs-api</artifactId>
</dependency>
<dependency>
<groupId>org.whispersystems.textsecure</groupId>
<artifactId>event-logger</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.whispersystems.textsecure</groupId>
<artifactId>websocket-resources</artifactId>
@@ -51,6 +82,11 @@
<artifactId>libsignal-server</artifactId>
</dependency>
<dependency>
<groupId>org.signal</groupId>
<artifactId>simple-grpc-runtime</artifactId>
</dependency>
<dependency>
<groupId>io.dropwizard</groupId>
<artifactId>dropwizard-core</artifactId>
@@ -65,7 +101,7 @@
</dependency>
<dependency>
<groupId>io.dropwizard</groupId>
<artifactId>dropwizard-db</artifactId>
<artifactId>dropwizard-http2</artifactId>
</dependency>
<dependency>
<groupId>io.dropwizard</groupId>
@@ -114,8 +150,8 @@
<artifactId>logback-core</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-access</artifactId>
<groupId>ch.qos.logback.access</groupId>
<artifactId>logback-access-common</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
@@ -150,19 +186,31 @@
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
</dependency>
<dependency>
<groupId>io.dropwizard</groupId>
<artifactId>dropwizard-testing</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-logback-appender-1.0</artifactId>
<!-- *all* opentelemetry-logback-appender versions are "alpha" despite the advanced version number -->
<version>2.19.0-alpha</version>
<exclusions>
<!-- incubator packages aren't included in the opentelemetry BOM, and we don't use them -->
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-instrumentation-api-incubator</artifactId>
</exclusion>
</exclusions>
</dependency>
@@ -189,12 +237,17 @@
<dependency>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>websocket-api</artifactId>
<artifactId>websocket-jetty-api</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlets</artifactId>
</dependency>
<dependency>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>websocket-jetty-client</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
@@ -204,11 +257,52 @@
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.28.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<exclusions>
<!-- our direct import of guava brings in a more recent version of failureaccess, so excluding it here -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>failureaccess</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.firebase</groupId>
<artifactId>firebase-admin</artifactId>
<version>9.1.1</version>
<version>${firebase-admin.version}</version>
<exclusions>
<!-- our direct import of guava brings in a more recent version of failureaccess, so excluding it here -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>failureaccess</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-firestore</artifactId>
<exclusions>
<!-- incubator packages aren't included in the opentelemetry BOM, and we don't use them -->
<exclusion>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-instrumentation-api-incubator</artifactId>
</exclusion>
<!-- our direct import of guava brings in a more recent version of failureaccess, so excluding it here -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>failureaccess</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
@@ -231,8 +325,7 @@
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
<scope>runtime</scope>
<artifactId>grpc-netty</artifactId>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
@@ -255,11 +348,7 @@
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-statsd</artifactId>
</dependency>
<dependency>
<groupId>org.coursera</groupId>
<artifactId>dropwizard-metrics-datadog</artifactId>
<artifactId>micrometer-registry-otlp</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
@@ -291,6 +380,19 @@
<artifactId>reactor-grpc-stub</artifactId>
</dependency>
<dependency>
<groupId>org.foundationdb</groupId>
<artifactId>fdb-java</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>apache-client</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>netty-nio-client</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sts</artifactId>
@@ -303,24 +405,10 @@
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>appconfig</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>appconfigdata</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>dynamodb-lock-client</artifactId>
<version>1.2.0</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
<version>1.4.0</version>
</dependency>
<dependency>
@@ -357,11 +445,30 @@
<artifactId>libphonenumber</artifactId>
</dependency>
<!-- Provides tools for mapping phone numbers to time zones, which is helpful for scheduling push notifications
during waking hours -->
<dependency>
<groupId>com.googlecode.libphonenumber</groupId>
<artifactId>geocoder</artifactId>
<version>3.14</version>
</dependency>
<dependency>
<groupId>net.sourceforge.argparse4j</groupId>
<artifactId>argparse4j</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-haproxy</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<classifier>linux-x86_64</classifier>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.test-framework</groupId>
<artifactId>jersey-test-framework-core</artifactId>
@@ -377,23 +484,6 @@
<groupId>org.glassfish.jersey.test-framework.providers</groupId>
<artifactId>jersey-test-framework-provider-grizzly2</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.almworks.sqlite4java</groupId>
<artifactId>sqlite4java</artifactId>
<version>1.0.392</version>
<scope>test</scope>
</dependency>
<dependency>
@@ -404,10 +494,6 @@
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core-micrometer</artifactId>
</dependency>
<dependency>
<groupId>io.vavr</groupId>
<artifactId>vavr</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
@@ -421,29 +507,33 @@
</dependency>
<dependency>
<groupId>org.signal</groupId>
<artifactId>embedded-redis</artifactId>
<groupId>com.redis</groupId>
<artifactId>testcontainers-redis</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.uuid</groupId>
<artifactId>java-uuid-generator</artifactId>
<version>4.0.1</version>
<version>${java-uuid-generator.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>DynamoDBLocal</artifactId>
<version>1.23.0</version>
<groupId>org.testcontainers</groupId>
<artifactId>localstack</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.github.ganadist.sqlite4java</groupId>
<artifactId>libsqlite4java-osx-aarch64</artifactId>
<version>1.0.392</version>
<type>dylib</type>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>earth.adi</groupId>
<artifactId>testcontainers-foundationdb</artifactId>
<scope>test</scope>
</dependency>
@@ -452,11 +542,6 @@
<artifactId>google-auth-library-oauth2-http</artifactId>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-recaptchaenterprise</artifactId>
</dependency>
<dependency>
<groupId>com.stripe</groupId>
<artifactId>stripe-java</artifactId>
@@ -470,7 +555,26 @@
<dependency>
<groupId>com.apollographql.apollo3</groupId>
<artifactId>apollo-api-jvm</artifactId>
<version>3.7.1</version>
<version>3.8.5</version>
<exclusions>
<exclusion>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
</exclusion>
<!-- conflicts with other users; resolved manually with explicit import -->
<exclusion>
<groupId>com.squareup.okio</groupId>
<artifactId>okio-jvm</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- to resolve conflicting imports from other dependencies -->
<dependency>
<groupId>com.squareup.okio</groupId>
<artifactId>okio-jvm</artifactId>
<version>3.15.0</version>
</dependency>
</dependencies>
@@ -480,10 +584,31 @@
<id>exclude-spam-filter</id>
<build>
<plugins>
<plugin>
<groupId>io.github.download-maven-plugin</groupId>
<artifactId>download-maven-plugin</artifactId>
<version>2.0.0</version>
<executions>
<execution>
<id>install-foundationdb-client-library</id>
<phase>prepare-package</phase>
<goals>
<goal>wget</goal>
</goals>
</execution>
</executions>
<configuration>
<url>https://github.com/apple/foundationdb/releases/download/${foundationdb.version}/libfdb_c.x86_64.so</url>
<outputDirectory>${project.build.directory}/jib-extra/usr/lib</outputDirectory>
<sha256>${foundationdb.client-library-sha256}</sha256>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
<filters>
@@ -518,7 +643,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor>
@@ -538,7 +662,6 @@
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<id>read-deploy-configuration</id>
@@ -567,6 +690,16 @@
<configuration>
<from>
<image>eclipse-temurin@sha256:${docker.image.sha256}</image>
<platforms>
<platform>
<architecture>amd64</architecture>
<os>linux</os>
</platform>
<platform>
<architecture>arm64</architecture>
<os>linux</os>
</platform>
</platforms>
</from>
<to>
<image>${docker.repo}:${project.version}</image>
@@ -578,6 +711,7 @@
<jvmFlag>-Djava.awt.headless=true</jvmFlag>
<jvmFlag>-Djdk.nio.maxCachedBufferSize=262144</jvmFlag>
<jvmFlag>-Dlog4j2.formatMsgNoLookups=true</jvmFlag>
<jvmFlag>-Djdk.tls.server.newSessionTicketCount=0</jvmFlag>
<jvmFlag>-XX:MaxRAMPercentage=75</jvmFlag>
<jvmFlag>-XX:+HeapDumpOnOutOfMemoryError</jvmFlag>
<jvmFlag>-XX:HeapDumpPath=/tmp/heapdump.bin</jvmFlag>
@@ -594,6 +728,9 @@
<includes>*.yml</includes>
<into>/usr/share/signal/</into>
</path>
<path>
<from>${project.build.directory}/jib-extra</from>
</path>
</paths>
</extraDirectories>
</configuration>
@@ -616,6 +753,30 @@
</plugins>
</build>
</profile>
<profile>
<id>test-server</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>start-test-server</id>
<phase>integration-test</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>org.whispersystems.textsecuregcm.LocalWhisperServerService</mainClass>
<classpathScope>test</classpathScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<build>
@@ -624,7 +785,7 @@
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>templating-maven-plugin</artifactId>
<version>1.0.0</version>
<version>3.0.0</version>
<executions>
<execution>
<id>filter-src</id>
@@ -632,16 +793,21 @@
<goal>filter-sources</goal>
</goals>
</execution>
<execution>
<id>filter-test-src</id>
<goals>
<goal>filter-test-sources</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<configuration>
<!-- work around PATCH not being a supported method on HttpUrlConnection -->
<argLine>--add-opens=java.base/java.net=ALL-UNNAMED</argLine>
<!-- add-opens: work around PATCH not being a supported method on HttpUrlConnection -->
<argLine>-javaagent:${org.mockito:mockito-core:jar} --add-opens=java.base/java.net=ALL-UNNAMED</argLine>
</configuration>
</plugin>
@@ -660,7 +826,6 @@
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>check-all-service-config</id>
@@ -668,15 +833,15 @@
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>org.whispersystems.textsecuregcm.CheckServiceConfigurations</mainClass>
<classpathScope>test</classpathScope>
<arguments>
<argument>${project.basedir}/config</argument>
</arguments>
</configuration>
</execution>
</executions>
<configuration>
<mainClass>org.whispersystems.textsecuregcm.CheckServiceConfigurations</mainClass>
<classpathScope>test</classpathScope>
<arguments>
<argument>${project.basedir}/config</argument>
</arguments>
</configuration>
</plugin>
<plugin>

View File

@@ -0,0 +1,20 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.storage;
public class FoundationDbVersion {
private static final String VERSION = "${foundationdb.version}";
private static final int API_VERSION = ${foundationdb.api-version};
public static String getFoundationDbVersion() {
return VERSION;
}
public static int getFoundationDbApiVersion() {
return API_VERSION;
}
}

View File

@@ -5,59 +5,66 @@
package org.whispersystems.textsecuregcm;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.dropwizard.Configuration;
import io.dropwizard.core.Configuration;
import jakarta.validation.Valid;
import jakarta.validation.constraints.NotNull;
import java.time.Duration;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import javax.validation.Valid;
import javax.validation.constraints.NotNull;
import org.whispersystems.textsecuregcm.attachments.TusConfiguration;
import org.whispersystems.textsecuregcm.configuration.AccountDatabaseCrawlerConfiguration;
import org.whispersystems.textsecuregcm.configuration.AdminEventLoggingConfiguration;
import org.whispersystems.textsecuregcm.configuration.ApnConfiguration;
import org.whispersystems.textsecuregcm.configuration.AppConfigConfiguration;
import org.whispersystems.textsecuregcm.configuration.ArtServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.AwsAttachmentsConfiguration;
import org.whispersystems.textsecuregcm.configuration.AppleAppStoreConfiguration;
import org.whispersystems.textsecuregcm.configuration.AppleDeviceCheckConfiguration;
import org.whispersystems.textsecuregcm.configuration.AwsCredentialsProviderFactory;
import org.whispersystems.textsecuregcm.configuration.BadgesConfiguration;
import org.whispersystems.textsecuregcm.configuration.BraintreeConfiguration;
import org.whispersystems.textsecuregcm.configuration.CallQualitySurveyConfiguration;
import org.whispersystems.textsecuregcm.configuration.Cdn3StorageManagerConfiguration;
import org.whispersystems.textsecuregcm.configuration.CdnConfiguration;
import org.whispersystems.textsecuregcm.configuration.CircuitBreakerConfiguration;
import org.whispersystems.textsecuregcm.configuration.ClientReleaseConfiguration;
import org.whispersystems.textsecuregcm.configuration.CommandStopListenerConfiguration;
import org.whispersystems.textsecuregcm.configuration.DogstatsdConfiguration;
import org.whispersystems.textsecuregcm.configuration.DefaultAwsCredentialsFactory;
import org.whispersystems.textsecuregcm.configuration.DeviceCheckConfiguration;
import org.whispersystems.textsecuregcm.configuration.DirectoryV2Configuration;
import org.whispersystems.textsecuregcm.configuration.DynamoDbClientConfiguration;
import org.whispersystems.textsecuregcm.configuration.DynamoDbClientFactory;
import org.whispersystems.textsecuregcm.configuration.DynamoDbTables;
import org.whispersystems.textsecuregcm.configuration.ExternalRequestFilterConfiguration;
import org.whispersystems.textsecuregcm.configuration.FaultTolerantRedisClientFactory;
import org.whispersystems.textsecuregcm.configuration.FaultTolerantRedisClusterFactory;
import org.whispersystems.textsecuregcm.configuration.FcmConfiguration;
import org.whispersystems.textsecuregcm.configuration.GcpAttachmentsConfiguration;
import org.whispersystems.textsecuregcm.configuration.GenericZkConfig;
import org.whispersystems.textsecuregcm.configuration.HCaptchaConfiguration;
import org.whispersystems.textsecuregcm.configuration.GooglePlayBillingConfiguration;
import org.whispersystems.textsecuregcm.configuration.GrpcConfiguration;
import org.whispersystems.textsecuregcm.configuration.IdlePrimaryDeviceReminderConfiguration;
import org.whispersystems.textsecuregcm.configuration.KeyTransparencyServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.LinkDeviceSecretConfiguration;
import org.whispersystems.textsecuregcm.configuration.MaxDeviceConfiguration;
import org.whispersystems.textsecuregcm.configuration.MessageByteLimitCardinalityEstimatorConfiguration;
import org.whispersystems.textsecuregcm.configuration.MessageCacheConfiguration;
import org.whispersystems.textsecuregcm.configuration.OneTimeDonationConfiguration;
import org.whispersystems.textsecuregcm.configuration.OpenTelemetryConfiguration;
import org.whispersystems.textsecuregcm.configuration.PagedSingleUseKEMPreKeyStoreConfiguration;
import org.whispersystems.textsecuregcm.configuration.PaymentsServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.RecaptchaConfiguration;
import org.whispersystems.textsecuregcm.configuration.RedisClusterConfiguration;
import org.whispersystems.textsecuregcm.configuration.RedisConfiguration;
import org.whispersystems.textsecuregcm.configuration.RegistrationServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.RegistrationServiceClientFactory;
import org.whispersystems.textsecuregcm.configuration.RemoteConfigConfiguration;
import org.whispersystems.textsecuregcm.configuration.ReportMessageConfiguration;
import org.whispersystems.textsecuregcm.configuration.SecureBackupServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.RetryConfiguration;
import org.whispersystems.textsecuregcm.configuration.S3ObjectMonitorFactory;
import org.whispersystems.textsecuregcm.configuration.SecureStorageServiceConfiguration;
import org.whispersystems.textsecuregcm.configuration.SecureValueRecovery2Configuration;
import org.whispersystems.textsecuregcm.configuration.SecureValueRecoveryConfiguration;
import org.whispersystems.textsecuregcm.configuration.ShortCodeExpanderConfiguration;
import org.whispersystems.textsecuregcm.configuration.SpamFilterConfiguration;
import org.whispersystems.textsecuregcm.configuration.StripeConfiguration;
import org.whispersystems.textsecuregcm.configuration.SubscriptionConfiguration;
import org.whispersystems.textsecuregcm.configuration.TurnSecretConfiguration;
import org.whispersystems.textsecuregcm.configuration.TlsKeyStoreConfiguration;
import org.whispersystems.textsecuregcm.configuration.TurnConfiguration;
import org.whispersystems.textsecuregcm.configuration.UnidentifiedDeliveryConfiguration;
import org.whispersystems.textsecuregcm.configuration.VirtualThreadConfiguration;
import org.whispersystems.textsecuregcm.configuration.ZkConfig;
import org.whispersystems.textsecuregcm.limits.RateLimiterConfig;
import org.whispersystems.websocket.configuration.WebSocketConfiguration;
/** @noinspection MismatchedQueryAndUpdateOfCollection, WeakerAccess */
@@ -66,7 +73,12 @@ public class WhisperServerConfiguration extends Configuration {
@NotNull
@Valid
@JsonProperty
private AdminEventLoggingConfiguration adminEventLoggingConfiguration;
private TlsKeyStoreConfiguration tlsKeyStore;
@NotNull
@Valid
@JsonProperty
AwsCredentialsProviderFactory awsCredentialsProvider = new DefaultAwsCredentialsFactory();
@NotNull
@Valid
@@ -81,18 +93,33 @@ public class WhisperServerConfiguration extends Configuration {
@NotNull
@Valid
@JsonProperty
private DynamoDbClientConfiguration dynamoDbClientConfiguration;
private GooglePlayBillingConfiguration googlePlayBilling;
@NotNull
@Valid
@JsonProperty
private AppleAppStoreConfiguration appleAppStore;
@NotNull
@Valid
@JsonProperty
private AppleDeviceCheckConfiguration appleDeviceCheck;
@NotNull
@Valid
@JsonProperty
private DeviceCheckConfiguration deviceCheck;
@NotNull
@Valid
@JsonProperty
private DynamoDbClientFactory dynamoDbClient;
@NotNull
@Valid
@JsonProperty
private DynamoDbTables dynamoDbTables;
@NotNull
@Valid
@JsonProperty
private AwsAttachmentsConfiguration awsAttachments;
@NotNull
@Valid
@JsonProperty
@@ -106,22 +133,22 @@ public class WhisperServerConfiguration extends Configuration {
@NotNull
@Valid
@JsonProperty
private DogstatsdConfiguration dogstatsd = new DogstatsdConfiguration();
private Cdn3StorageManagerConfiguration cdn3StorageManager;
@NotNull
@Valid
@JsonProperty
private RedisClusterConfiguration cacheCluster;
private OpenTelemetryConfiguration openTelemetry;
@NotNull
@Valid
@JsonProperty
private RedisConfiguration pubsub;
private FaultTolerantRedisClusterFactory cacheCluster;
@NotNull
@Valid
@JsonProperty
private RedisClusterConfiguration metricsCluster;
private FaultTolerantRedisClientFactory pubsub;
@NotNull
@Valid
@@ -131,48 +158,33 @@ public class WhisperServerConfiguration extends Configuration {
@NotNull
@Valid
@JsonProperty
private SecureValueRecovery2Configuration svr2;
private SecureValueRecoveryConfiguration svr2;
@NotNull
@Valid
@JsonProperty
private AccountDatabaseCrawlerConfiguration accountDatabaseCrawler;
private SecureValueRecoveryConfiguration svrb;
@NotNull
@Valid
@JsonProperty
private RedisClusterConfiguration pushSchedulerCluster;
private FaultTolerantRedisClusterFactory pushSchedulerCluster;
@NotNull
@Valid
@JsonProperty
private RedisClusterConfiguration rateLimitersCluster;
private FaultTolerantRedisClusterFactory rateLimitersCluster;
@NotNull
@Valid
@JsonProperty
private MessageCacheConfiguration messageCache;
@NotNull
@Valid
@JsonProperty
private RedisClusterConfiguration clientPresenceCluster;
@Valid
@NotNull
@JsonProperty
private Set<String> testDevices = new HashSet<>();
@Valid
@NotNull
@JsonProperty
private List<MaxDeviceConfiguration> maxDevices = new LinkedList<>();
@Valid
@NotNull
@JsonProperty
private Map<String, RateLimiterConfig> limits = new HashMap<>();
@Valid
@NotNull
@JsonProperty
@@ -193,16 +205,6 @@ public class WhisperServerConfiguration extends Configuration {
@JsonProperty
private UnidentifiedDeliveryConfiguration unidentifiedDelivery;
@Valid
@NotNull
@JsonProperty
private RecaptchaConfiguration recaptcha;
@Valid
@NotNull
@JsonProperty
private HCaptchaConfiguration hCaptcha;
@Valid
@NotNull
@JsonProperty
@@ -213,21 +215,11 @@ public class WhisperServerConfiguration extends Configuration {
@JsonProperty
private SecureStorageServiceConfiguration storageService;
@Valid
@NotNull
@JsonProperty
private SecureBackupServiceConfiguration backupService;
@Valid
@NotNull
@JsonProperty
private PaymentsServiceConfiguration paymentsService;
@Valid
@NotNull
@JsonProperty
private ArtServiceConfiguration artService;
@Valid
@NotNull
@JsonProperty
@@ -236,7 +228,12 @@ public class WhisperServerConfiguration extends Configuration {
@Valid
@NotNull
@JsonProperty
private GenericZkConfig genericZkConfig;
private GenericZkConfig callingZkConfig;
@Valid
@NotNull
@JsonProperty
private GenericZkConfig backupsZkConfig;
@Valid
@NotNull
@@ -246,7 +243,7 @@ public class WhisperServerConfiguration extends Configuration {
@Valid
@NotNull
@JsonProperty
private AppConfigConfiguration appConfig;
private S3ObjectMonitorFactory dynamicConfig;
@Valid
@NotNull
@@ -263,6 +260,11 @@ public class WhisperServerConfiguration extends Configuration {
@NotNull
private OneTimeDonationConfiguration oneTimeDonations;
@Valid
@JsonProperty
@NotNull
private PagedSingleUseKEMPreKeyStoreConfiguration pagedSingleUseKEMPreKeyStore;
@Valid
@NotNull
@JsonProperty
@@ -270,28 +272,23 @@ public class WhisperServerConfiguration extends Configuration {
@Valid
@JsonProperty
private SpamFilterConfiguration spamFilterConfiguration;
private SpamFilterConfiguration spamFilter;
@Valid
@NotNull
@JsonProperty
private RegistrationServiceConfiguration registrationService;
private RegistrationServiceClientFactory registrationService;
@Valid
@NotNull
@JsonProperty
private TurnSecretConfiguration turn;
private TurnConfiguration turn;
@Valid
@NotNull
@JsonProperty
private TusConfiguration tus;
@Valid
@NotNull
@JsonProperty
private int grpcPort;
@Valid
@NotNull
@JsonProperty
@@ -305,15 +302,62 @@ public class WhisperServerConfiguration extends Configuration {
@Valid
@NotNull
@JsonProperty
private CommandStopListenerConfiguration commandStopListener;
private LinkDeviceSecretConfiguration linkDevice;
@Valid
@NotNull
@JsonProperty
private LinkDeviceSecretConfiguration linkDevice;
private VirtualThreadConfiguration virtualThread = new VirtualThreadConfiguration();
public AdminEventLoggingConfiguration getAdminEventLoggingConfiguration() {
return adminEventLoggingConfiguration;
@Valid
@NotNull
@JsonProperty
private ExternalRequestFilterConfiguration externalRequestFilter;
@Valid
@NotNull
@JsonProperty
private KeyTransparencyServiceConfiguration keyTransparencyService;
@JsonProperty
private boolean logMessageDeliveryLoops;
@JsonProperty
private IdlePrimaryDeviceReminderConfiguration idlePrimaryDeviceReminder =
new IdlePrimaryDeviceReminderConfiguration(Duration.ofDays(30));
@JsonProperty
private Map<String, @Valid CircuitBreakerConfiguration> circuitBreakers = Collections.emptyMap();
@JsonProperty
private Map<String, @Valid RetryConfiguration> retries = Collections.emptyMap();
@JsonProperty
@Valid
@NotNull
private RetryConfiguration generalRedisRetry = new RetryConfiguration();
@NotNull
@Valid
@JsonProperty
private GrpcConfiguration grpc;
@Valid
@NotNull
@JsonProperty
private S3ObjectMonitorFactory asnTable;
@Valid
@NotNull
@JsonProperty
private CallQualitySurveyConfiguration callQualitySurvey;
public TlsKeyStoreConfiguration getTlsKeyStoreConfiguration() {
return tlsKeyStore;
}
public AwsCredentialsProviderFactory getAwsCredentialsConfiguration() {
return awsCredentialsProvider;
}
public StripeConfiguration getStripe() {
@@ -324,22 +368,30 @@ public class WhisperServerConfiguration extends Configuration {
return braintree;
}
public DynamoDbClientConfiguration getDynamoDbClientConfiguration() {
return dynamoDbClientConfiguration;
public GooglePlayBillingConfiguration getGooglePlayBilling() {
return googlePlayBilling;
}
public AppleAppStoreConfiguration getAppleAppStore() {
return appleAppStore;
}
public AppleDeviceCheckConfiguration getAppleDeviceCheck() {
return appleDeviceCheck;
}
public DeviceCheckConfiguration getDeviceCheck() {
return deviceCheck;
}
public DynamoDbClientFactory getDynamoDbClientConfiguration() {
return dynamoDbClient;
}
public DynamoDbTables getDynamoDbTables() {
return dynamoDbTables;
}
public RecaptchaConfiguration getRecaptchaConfiguration() {
return recaptcha;
}
public HCaptchaConfiguration getHCaptchaConfiguration() {
return hCaptcha;
}
public ShortCodeExpanderConfiguration getShortCodeRetrieverConfiguration() {
return shortCode;
}
@@ -348,28 +400,24 @@ public class WhisperServerConfiguration extends Configuration {
return webSocket;
}
public AwsAttachmentsConfiguration getAwsAttachmentsConfiguration() {
return awsAttachments;
}
public GcpAttachmentsConfiguration getGcpAttachmentsConfiguration() {
return gcpAttachments;
}
public RedisClusterConfiguration getCacheClusterConfiguration() {
public FaultTolerantRedisClusterFactory getCacheClusterConfiguration() {
return cacheCluster;
}
public RedisConfiguration getPubsubCacheConfiguration() {
public FaultTolerantRedisClientFactory getRedisPubSubConfiguration() {
return pubsub;
}
public RedisClusterConfiguration getMetricsClusterConfiguration() {
return metricsCluster;
public SecureValueRecoveryConfiguration getSvr2Configuration() {
return svr2;
}
public SecureValueRecovery2Configuration getSvr2Configuration() {
return svr2;
public SecureValueRecoveryConfiguration getSvrbConfiguration() {
return svrb;
}
public DirectoryV2Configuration getDirectoryV2Configuration() {
@@ -380,30 +428,18 @@ public class WhisperServerConfiguration extends Configuration {
return storageService;
}
public AccountDatabaseCrawlerConfiguration getAccountDatabaseCrawlerConfiguration() {
return accountDatabaseCrawler;
}
public MessageCacheConfiguration getMessageCacheConfiguration() {
return messageCache;
}
public RedisClusterConfiguration getClientPresenceClusterConfiguration() {
return clientPresenceCluster;
}
public RedisClusterConfiguration getPushSchedulerCluster() {
public FaultTolerantRedisClusterFactory getPushSchedulerCluster() {
return pushSchedulerCluster;
}
public RedisClusterConfiguration getRateLimitersCluster() {
public FaultTolerantRedisClusterFactory getRateLimitersCluster() {
return rateLimitersCluster;
}
public Map<String, RateLimiterConfig> getLimitsConfiguration() {
return limits;
}
public FcmConfiguration getFcmConfiguration() {
return fcm;
}
@@ -416,18 +452,18 @@ public class WhisperServerConfiguration extends Configuration {
return cdn;
}
public DogstatsdConfiguration getDatadogConfiguration() {
return dogstatsd;
public Cdn3StorageManagerConfiguration getCdn3StorageManagerConfiguration() {
return cdn3StorageManager;
}
public OpenTelemetryConfiguration getOpenTelemetryConfiguration() {
return openTelemetry;
}
public UnidentifiedDeliveryConfiguration getDeliveryCertificate() {
return unidentifiedDelivery;
}
public Set<String> getTestDevices() {
return testDevices;
}
public Map<String, Integer> getMaxDevices() {
Map<String, Integer> results = new HashMap<>();
@@ -439,32 +475,28 @@ public class WhisperServerConfiguration extends Configuration {
return results;
}
public SecureBackupServiceConfiguration getSecureBackupServiceConfiguration() {
return backupService;
}
public PaymentsServiceConfiguration getPaymentsServiceConfiguration() {
return paymentsService;
}
public ArtServiceConfiguration getArtServiceConfiguration() {
return artService;
}
public ZkConfig getZkConfig() {
return zkConfig;
}
public GenericZkConfig getGenericZkConfig() {
return genericZkConfig;
public GenericZkConfig getCallingZkConfig() {
return callingZkConfig;
}
public GenericZkConfig getBackupsZkConfig() {
return backupsZkConfig;
}
public RemoteConfigConfiguration getRemoteConfigConfiguration() {
return remoteConfig;
}
public AppConfigConfiguration getAppConfig() {
return appConfig;
public S3ObjectMonitorFactory getDynamicConfig() {
return dynamicConfig;
}
public BadgesConfiguration getBadges() {
@@ -479,19 +511,23 @@ public class WhisperServerConfiguration extends Configuration {
return oneTimeDonations;
}
public PagedSingleUseKEMPreKeyStoreConfiguration getPagedSingleUseKEMPreKeyStore() {
return pagedSingleUseKEMPreKeyStore;
}
public ReportMessageConfiguration getReportMessageConfiguration() {
return reportMessage;
}
public SpamFilterConfiguration getSpamFilterConfiguration() {
return spamFilterConfiguration;
return spamFilter;
}
public RegistrationServiceConfiguration getRegistrationServiceConfiguration() {
public RegistrationServiceClientFactory getRegistrationServiceConfiguration() {
return registrationService;
}
public TurnSecretConfiguration getTurnSecretConfiguration() {
public TurnConfiguration getTurnConfiguration() {
return turn;
}
@@ -499,10 +535,6 @@ public class WhisperServerConfiguration extends Configuration {
return tus;
}
public int getGrpcPort() {
return grpcPort;
}
public ClientReleaseConfiguration getClientReleaseConfiguration() {
return clientRelease;
}
@@ -511,11 +543,51 @@ public class WhisperServerConfiguration extends Configuration {
return messageByteLimitCardinalityEstimator;
}
public CommandStopListenerConfiguration getCommandStopListener() {
return commandStopListener;
}
public LinkDeviceSecretConfiguration getLinkDeviceSecretConfiguration() {
return linkDevice;
}
public VirtualThreadConfiguration getVirtualThreadConfiguration() {
return virtualThread;
}
public ExternalRequestFilterConfiguration getExternalRequestFilterConfiguration() {
return externalRequestFilter;
}
public KeyTransparencyServiceConfiguration getKeyTransparencyServiceConfiguration() {
return keyTransparencyService;
}
public boolean logMessageDeliveryLoops() {
return logMessageDeliveryLoops;
}
public IdlePrimaryDeviceReminderConfiguration idlePrimaryDeviceReminderConfiguration() {
return idlePrimaryDeviceReminder;
}
public Map<String, CircuitBreakerConfiguration> getCircuitBreakerConfigurations() {
return circuitBreakers;
}
public Map<String, RetryConfiguration> getRetryConfigurations() {
return retries;
}
public RetryConfiguration getGeneralRedisRetryConfiguration() {
return generalRedisRetry;
}
public GrpcConfiguration getGrpc() {
return grpc;
}
public S3ObjectMonitorFactory getAsnTableConfiguration() {
return asnTable;
}
public CallQualitySurveyConfiguration getCallQualitySurveyConfiguration() {
return callQualitySurvey;
}
}

View File

@@ -0,0 +1,18 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.asn;
import static java.util.Objects.requireNonNull;
import com.google.i18n.phonenumbers.PhoneNumberUtil;
import javax.annotation.Nonnull;
public record AsnInfo(long asn, @Nonnull String regionCode) {
public AsnInfo {
requireNonNull(regionCode, "regionCode must not be null");
}
}

View File

@@ -0,0 +1,21 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.asn;
import java.util.Optional;
import javax.annotation.Nonnull;
public interface AsnInfoProvider {
/// Gets ASN information for an IP address.
///
/// @param ipString a string representation of an IP address
///
/// @return ASN information for the given IP address or empty if no ASN information was found for the given IP address
Optional<AsnInfo> lookup(@Nonnull String ipString);
AsnInfoProvider EMPTY = _ -> Optional.empty();
}

View File

@@ -0,0 +1,165 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.asn;
import static java.util.Objects.requireNonNull;
import com.google.common.annotations.VisibleForTesting;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.lang.invoke.MethodHandles;
import java.math.BigInteger;
import java.net.Inet4Address;
import java.net.Inet6Address;
import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.util.HashMap;
import java.util.Map;
import java.util.NavigableMap;
import java.util.Optional;
import java.util.TreeMap;
import java.util.zip.GZIPInputStream;
import javax.annotation.Nonnull;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVParser;
import org.apache.commons.csv.CSVRecord;
import org.apache.commons.lang3.Validate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* {@code AsnInfoProvider} implementation that supports both IPv4 and IPv6.
*/
public class AsnInfoProviderImpl implements AsnInfoProvider {
private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
@Nonnull
private final NavigableMap<Long, AsnRange<Long>> asnBlocksByFirstIpv4;
@Nonnull
private final NavigableMap<BigInteger, AsnRange<BigInteger>> asnBlocksByFirstIpv6;
/**
* Creates an instance of {@code AsnInfoProviderImpl} using data from <a href="https://iptoasn.com/">iptoasn.com</a>.
* @param tsvGzInputStream gzip input stream representing the data.
*/
@Nonnull
public static AsnInfoProviderImpl fromTsvGz(@Nonnull final InputStream tsvGzInputStream) {
try (final GZIPInputStream inputStream = new GZIPInputStream(tsvGzInputStream)) {
return fromTsv(inputStream);
} catch (final IOException e) {
log.error("failed to ungzip the input stream", e);
throw new RuntimeException(e);
}
}
/**
* Creates an instance of {@code AsnInfoProviderImpl} using data from <a href="https://iptoasn.com/">iptoasn.com</a>.
* @param tsvInputStream input stream representing the data.
*/
@Nonnull
public static AsnInfoProviderImpl fromTsv(@Nonnull final InputStream tsvInputStream) {
try (final InputStreamReader tsvReader = new InputStreamReader(tsvInputStream)) {
final NavigableMap<Long, AsnRange<Long>> ip4asns = new TreeMap<>();
final NavigableMap<BigInteger, AsnRange<BigInteger>> ip6asns = new TreeMap<>();
final Map<Long, AsnInfo> asnInfoCache = new HashMap<>();
try (final CSVParser csvParser = CSVFormat.TDF.parse(tsvReader)) {
for (final CSVRecord record : csvParser) {
// format:
// range_start_ip_string range_end_ip_string AS_number country_code AS_description
final InetAddress startIp = InetAddress.getByName(record.get(0));
final InetAddress endIp = InetAddress.getByName(record.get(1));
final long asn = Long.parseLong(record.get(2));
final String regionCode = record.get(3);
// country code should be the same for any ASN, so we're caching AsnInfo objects
// not to have multiple instances with the same values
final AsnInfo asnInfo = asnInfoCache.computeIfAbsent(asn, k -> new AsnInfo(asn, regionCode));
if (!regionCode.equals(asnInfo.regionCode())) {
log.warn("ASN {} mapped to country codes {} and {}", asn, regionCode, asnInfo.regionCode());
}
// IPv4
if (startIp instanceof Inet4Address) {
final AsnRange<Long> asnRange = new AsnRange<>(
ip4BytesToLong((Inet4Address) startIp),
ip4BytesToLong((Inet4Address) endIp),
asnInfo
);
ip4asns.put(asnRange.from(), asnRange);
}
// IPv6
if (startIp instanceof Inet6Address) {
final AsnRange<BigInteger> asnRange = new AsnRange<>(
ip6BytesToBigInteger((Inet6Address) startIp),
ip6BytesToBigInteger((Inet6Address) endIp),
asnInfo
);
ip6asns.put(asnRange.from(), asnRange);
}
}
}
return new AsnInfoProviderImpl(ip4asns, ip6asns);
} catch (final Exception e) {
throw new RuntimeException(e);
}
}
public AsnInfoProviderImpl(
@Nonnull final NavigableMap<Long, AsnRange<Long>> asnBlocksByFirstIpv4,
@Nonnull final NavigableMap<BigInteger, AsnRange<BigInteger>> asnBlocksByFirstIpv6) {
this.asnBlocksByFirstIpv4 = requireNonNull(asnBlocksByFirstIpv4);
this.asnBlocksByFirstIpv6 = requireNonNull(asnBlocksByFirstIpv6);
}
@Nonnull
@Override
public Optional<AsnInfo> lookup(@Nonnull final String ipString) {
try {
final InetAddress address = InetAddress.getByName(ipString);
if (address instanceof Inet4Address ip4) {
final Long key = ip4BytesToLong(ip4);
return lookupInMap(asnBlocksByFirstIpv4, key);
}
if (address instanceof Inet6Address ip6) {
final BigInteger key = ip6BytesToBigInteger(ip6);
return lookupInMap(asnBlocksByFirstIpv6, key);
}
// safety net, should never happen
log.warn("Unknown InetAddress implementation: {}", address.getClass().getName());
} catch (final Exception e) {
log.error("Could not resolve ASN for IP string {}", ipString);
}
return Optional.empty();
}
@VisibleForTesting
protected static long ip4BytesToLong(@Nonnull final Inet4Address address) {
final byte[] arr = address.getAddress();
Validate.isTrue(arr.length == 4);
return Integer.toUnsignedLong(ByteBuffer.wrap(arr).getInt());
}
@VisibleForTesting
protected static BigInteger ip6BytesToBigInteger(@Nonnull final Inet6Address address) {
final byte[] arr = address.getAddress();
Validate.isTrue(arr.length == 16);
return new BigInteger(1, arr);
}
@Nonnull
private static <T extends Comparable<T>> Optional<AsnInfo> lookupInMap(
@Nonnull final NavigableMap<T, AsnRange<T>> map,
@Nonnull final T key) {
return Optional.ofNullable(map.floorEntry(key))
.filter(e -> e.getValue().contains(key) && e.getValue().asnInfo().asn() != 0)
.map(e -> e.getValue().asnInfo());
}
}

View File

@@ -0,0 +1,28 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.asn;
import static java.util.Objects.requireNonNull;
import javax.annotation.Nonnull;
import org.apache.commons.lang3.Validate;
public record AsnRange<T extends Comparable<T>>(@Nonnull T from,
@Nonnull T to,
@Nonnull AsnInfo asnInfo) {
public AsnRange {
requireNonNull(from);
requireNonNull(to);
requireNonNull(asnInfo);
Validate.isTrue(from.compareTo(to) <= 0);
}
boolean contains(@Nonnull final T element) {
requireNonNull(element);
return from.compareTo(element) <= 0
&& element.compareTo(to) <= 0;
}
}

View File

@@ -5,9 +5,9 @@
package org.whispersystems.textsecuregcm.attachments;
import jakarta.validation.constraints.NotEmpty;
import org.whispersystems.textsecuregcm.configuration.secrets.SecretBytes;
import org.whispersystems.textsecuregcm.util.ExactlySize;
import javax.validation.constraints.NotEmpty;
public record TusConfiguration(
@ExactlySize(32) SecretBytes userAuthenticationTokenSharedSecret,

View File

@@ -1,16 +0,0 @@
/*
* Copyright 2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
public interface AccountAndAuthenticatedDeviceHolder {
Account getAccount();
Device getAuthenticatedDevice();
}

View File

@@ -1,25 +1,161 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* Copyright 2013 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import static org.whispersystems.textsecuregcm.metrics.MetricsUtil.name;
import com.google.common.annotations.VisibleForTesting;
import io.dropwizard.auth.Authenticator;
import io.dropwizard.auth.basic.BasicCredentials;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Tags;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.Optional;
import org.whispersystems.textsecuregcm.experiment.ExperimentEnrollmentManager;
import java.util.UUID;
import org.apache.commons.lang3.StringUtils;
import org.whispersystems.textsecuregcm.identity.IdentityType;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.util.Pair;
import org.whispersystems.textsecuregcm.util.Util;
public class AccountAuthenticator extends BaseAccountAuthenticator implements
Authenticator<BasicCredentials, AuthenticatedAccount> {
public class AccountAuthenticator implements Authenticator<BasicCredentials, AuthenticatedDevice> {
private static final String AUTHENTICATION_COUNTER_NAME = name(AccountAuthenticator.class, "authentication");
private static final String AUTHENTICATION_SUCCEEDED_TAG_NAME = "succeeded";
private static final String AUTHENTICATION_FAILURE_REASON_TAG_NAME = "reason";
private static final String DAYS_SINCE_LAST_SEEN_DISTRIBUTION_NAME = name(AccountAuthenticator.class, "daysSinceLastSeen");
private static final String IS_PRIMARY_DEVICE_TAG = "isPrimary";
private static final Counter OLD_TOKEN_VERSION_COUNTER =
Metrics.counter(name(AccountAuthenticator.class, "oldTokenVersionCounter"));
@VisibleForTesting
static final char DEVICE_ID_SEPARATOR = '.';
private final AccountsManager accountsManager;
private final Clock clock;
public AccountAuthenticator(AccountsManager accountsManager) {
super(accountsManager);
this(accountsManager, Clock.systemUTC());
}
@VisibleForTesting
public AccountAuthenticator(AccountsManager accountsManager, Clock clock) {
this.accountsManager = accountsManager;
this.clock = clock;
}
static Pair<String, Byte> getIdentifierAndDeviceId(final String basicUsername) {
final String identifier;
final byte deviceId;
final int deviceIdSeparatorIndex = basicUsername.indexOf(DEVICE_ID_SEPARATOR);
if (deviceIdSeparatorIndex == -1) {
identifier = basicUsername;
deviceId = Device.PRIMARY_ID;
} else {
identifier = basicUsername.substring(0, deviceIdSeparatorIndex);
deviceId = Byte.parseByte(basicUsername.substring(deviceIdSeparatorIndex + 1));
}
return new Pair<>(identifier, deviceId);
}
@Override
public Optional<AuthenticatedAccount> authenticate(BasicCredentials basicCredentials) {
return super.authenticate(basicCredentials, true);
public Optional<AuthenticatedDevice> authenticate(BasicCredentials basicCredentials) {
boolean succeeded = false;
String failureReason = null;
try {
final UUID accountUuid;
final byte deviceId;
{
final Pair<String, Byte> identifierAndDeviceId = getIdentifierAndDeviceId(basicCredentials.getUsername());
accountUuid = UUID.fromString(identifierAndDeviceId.first());
deviceId = identifierAndDeviceId.second();
}
Optional<Account> account = accountsManager.getByAccountIdentifier(accountUuid);
if (account.isEmpty()) {
failureReason = "noSuchAccount";
return Optional.empty();
}
Optional<Device> device = account.get().getDevice(deviceId);
if (device.isEmpty()) {
failureReason = "noSuchDevice";
return Optional.empty();
}
SaltedTokenHash deviceSaltedTokenHash = device.get().getAuthTokenHash();
if (deviceSaltedTokenHash.verify(basicCredentials.getPassword())) {
succeeded = true;
Account authenticatedAccount = updateLastSeen(account.get(), device.get());
if (deviceSaltedTokenHash.getVersion() != SaltedTokenHash.CURRENT_VERSION) {
OLD_TOKEN_VERSION_COUNTER.increment();
authenticatedAccount = accountsManager.updateDeviceAuthentication(
authenticatedAccount,
device.get(),
SaltedTokenHash.generateFor(basicCredentials.getPassword())); // new credentials have current version
}
return Optional.of(new AuthenticatedDevice(authenticatedAccount.getIdentifier(IdentityType.ACI),
device.get().getId(),
Instant.ofEpochMilli(authenticatedAccount.getPrimaryDevice().getLastSeen())));
} else {
failureReason = "incorrectPassword";
return Optional.empty();
}
} catch (IllegalArgumentException | InvalidAuthorizationHeaderException iae) {
failureReason = "invalidHeader";
return Optional.empty();
} finally {
Tags tags = Tags.of(
AUTHENTICATION_SUCCEEDED_TAG_NAME, String.valueOf(succeeded));
if (StringUtils.isNotBlank(failureReason)) {
tags = tags.and(AUTHENTICATION_FAILURE_REASON_TAG_NAME, failureReason);
}
Metrics.counter(AUTHENTICATION_COUNTER_NAME, tags).increment();
}
}
@VisibleForTesting
public Account updateLastSeen(Account account, Device device) {
// compute a non-negative integer between 0 and 86400.
long n = Util.ensureNonNegativeLong(account.getUuid().getLeastSignificantBits());
final long lastSeenOffsetSeconds = n % ChronoUnit.DAYS.getDuration().toSeconds();
// produce a truncated timestamp which is either today at UTC midnight
// or yesterday at UTC midnight, based on per-user randomized offset used.
final long todayInMillisWithOffset = Util.todayInMillisGivenOffsetFromNow(clock,
Duration.ofSeconds(lastSeenOffsetSeconds).negated());
// only update the device's last seen time when it falls behind the truncated timestamp.
// this ensures a few things:
// (1) each account will only update last-seen at most once per day
// (2) these updates will occur throughout the day rather than all occurring at UTC midnight.
if (device.getLastSeen() < todayInMillisWithOffset) {
Metrics.summary(DAYS_SINCE_LAST_SEEN_DISTRIBUTION_NAME, IS_PRIMARY_DEVICE_TAG, String.valueOf(device.isPrimary()))
.record(Duration.ofMillis(todayInMillisWithOffset - device.getLastSeen()).toDays());
return accountsManager.updateDeviceLastSeen(account, device, Util.todayInMillis(clock));
}
return account;
}
}

View File

@@ -5,9 +5,9 @@
package org.whispersystems.textsecuregcm.auth;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response;
import java.util.Base64;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
public class Anonymous {

View File

@@ -1,100 +0,0 @@
/*
* Copyright 2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import com.google.common.annotations.VisibleForTesting;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.stream.Collectors;
import org.glassfish.jersey.server.ContainerRequest;
import org.glassfish.jersey.server.monitoring.RequestEvent;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.util.Pair;
/**
* This {@link WebsocketRefreshRequirementProvider} observes intra-request changes in {@link Account#isEnabled()} and
* {@link Device#isEnabled()}.
* <p>
* If a change in {@link Account#isEnabled()} or any associated {@link Device#isEnabled()} is observed, then any active
* WebSocket connections for the account must be closed in order for clients to get a refreshed
* {@link io.dropwizard.auth.Auth} object with a current device list.
*
* @see AuthenticatedAccount
* @see DisabledPermittedAuthenticatedAccount
*/
public class AuthEnablementRefreshRequirementProvider implements WebsocketRefreshRequirementProvider {
private final AccountsManager accountsManager;
private static final Logger logger = LoggerFactory.getLogger(AuthEnablementRefreshRequirementProvider.class);
private static final String ACCOUNT_UUID = AuthEnablementRefreshRequirementProvider.class.getName() + ".accountUuid";
private static final String DEVICES_ENABLED = AuthEnablementRefreshRequirementProvider.class.getName() + ".devicesEnabled";
public AuthEnablementRefreshRequirementProvider(final AccountsManager accountsManager) {
this.accountsManager = accountsManager;
}
@VisibleForTesting
static Map<Long, Boolean> buildDevicesEnabledMap(final Account account) {
return account.getDevices().stream().collect(Collectors.toMap(Device::getId, Device::isEnabled));
}
@Override
public void handleRequestFiltered(final RequestEvent requestEvent) {
if (requestEvent.getUriInfo().getMatchedResourceMethod().getInvocable().getHandlingMethod().getAnnotation(ChangesDeviceEnabledState.class) != null) {
// The authenticated principal, if any, will be available after filters have run.
// Now that the account is known, capture a snapshot of `isEnabled` for the account's devices before carrying out
// the requests business logic.
ContainerRequestUtil.getAuthenticatedAccount(requestEvent.getContainerRequest()).ifPresent(account ->
setAccount(requestEvent.getContainerRequest(), account));
}
}
public static void setAccount(final ContainerRequest containerRequest, final Account account) {
containerRequest.setProperty(ACCOUNT_UUID, account.getUuid());
containerRequest.setProperty(DEVICES_ENABLED, buildDevicesEnabledMap(account));
}
@Override
public List<Pair<UUID, Long>> handleRequestFinished(final RequestEvent requestEvent) {
// Now that the request is finished, check whether `isEnabled` changed for any of the devices. If the value did
// change or if a devices was added or removed, all devices must disconnect and reauthenticate.
if (requestEvent.getContainerRequest().getProperty(DEVICES_ENABLED) != null) {
@SuppressWarnings("unchecked") final Map<Long, Boolean> initialDevicesEnabled =
(Map<Long, Boolean>) requestEvent.getContainerRequest().getProperty(DEVICES_ENABLED);
return accountsManager.getByAccountIdentifier((UUID) requestEvent.getContainerRequest().getProperty(ACCOUNT_UUID)).map(account -> {
final Set<Long> deviceIdsToDisplace;
final Map<Long, Boolean> currentDevicesEnabled = buildDevicesEnabledMap(account);
if (!initialDevicesEnabled.equals(currentDevicesEnabled)) {
deviceIdsToDisplace = new HashSet<>(initialDevicesEnabled.keySet());
deviceIdsToDisplace.addAll(currentDevicesEnabled.keySet());
} else {
deviceIdsToDisplace = Collections.emptySet();
}
return deviceIdsToDisplace.stream()
.map(deviceId -> new Pair<>(account.getUuid(), deviceId))
.collect(Collectors.toList());
}).orElseGet(() -> {
logger.error("Request had account, but it is no longer present");
return Collections.emptyList();
});
} else
return Collections.emptyList();
}
}

View File

@@ -1,44 +0,0 @@
/*
* Copyright 2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.security.Principal;
import java.util.function.Supplier;
import javax.security.auth.Subject;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.util.Pair;
public class AuthenticatedAccount implements Principal, AccountAndAuthenticatedDeviceHolder {
private final Supplier<Pair<Account, Device>> accountAndDevice;
public AuthenticatedAccount(final Supplier<Pair<Account, Device>> accountAndDevice) {
this.accountAndDevice = accountAndDevice;
}
@Override
public Account getAccount() {
return accountAndDevice.get().first();
}
@Override
public Device getAuthenticatedDevice() {
return accountAndDevice.get().second();
}
// Principal implementation
@Override
public String getName() {
return null;
}
@Override
public boolean implies(final Subject subject) {
return false;
}
}

View File

@@ -0,0 +1,20 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import org.signal.libsignal.zkgroup.backups.BackupCredentialType;
import org.signal.libsignal.zkgroup.backups.BackupLevel;
import org.whispersystems.textsecuregcm.util.ua.UserAgent;
import javax.annotation.Nullable;
public record AuthenticatedBackupUser(
byte[] backupId,
BackupCredentialType credentialType,
BackupLevel backupLevel,
String backupDir,
String mediaDir,
@Nullable UserAgent userAgent) {
}

View File

@@ -0,0 +1,25 @@
/*
* Copyright 2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.security.Principal;
import java.time.Instant;
import java.util.UUID;
import javax.security.auth.Subject;
public record AuthenticatedDevice(UUID accountIdentifier, byte deviceId, Instant primaryDeviceLastSeen)
implements Principal {
@Override
public String getName() {
return null;
}
@Override
public boolean implies(final Subject subject) {
return false;
}
}

View File

@@ -1,173 +0,0 @@
/*
* Copyright 2013 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import static com.codahale.metrics.MetricRegistry.name;
import com.google.common.annotations.VisibleForTesting;
import io.dropwizard.auth.basic.BasicCredentials;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Tags;
import java.time.Clock;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.Optional;
import java.util.UUID;
import org.apache.commons.lang3.StringUtils;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.storage.RefreshingAccountAndDeviceSupplier;
import org.whispersystems.textsecuregcm.util.Pair;
import org.whispersystems.textsecuregcm.util.Util;
public class BaseAccountAuthenticator {
private static final String AUTHENTICATION_COUNTER_NAME = name(BaseAccountAuthenticator.class, "authentication");
private static final String ENABLED_NOT_REQUIRED_AUTHENTICATION_COUNTER_NAME = name(BaseAccountAuthenticator.class,
"enabledNotRequiredAuthentication");
private static final String AUTHENTICATION_SUCCEEDED_TAG_NAME = "succeeded";
private static final String AUTHENTICATION_FAILURE_REASON_TAG_NAME = "reason";
private static final String ENABLED_TAG_NAME = "enabled";
private static final String DAYS_SINCE_LAST_SEEN_DISTRIBUTION_NAME = name(BaseAccountAuthenticator.class, "daysSinceLastSeen");
private static final String IS_PRIMARY_DEVICE_TAG = "isPrimary";
@VisibleForTesting
static final char DEVICE_ID_SEPARATOR = '.';
private final AccountsManager accountsManager;
private final Clock clock;
public BaseAccountAuthenticator(AccountsManager accountsManager) {
this(accountsManager, Clock.systemUTC());
}
@VisibleForTesting
public BaseAccountAuthenticator(AccountsManager accountsManager, Clock clock) {
this.accountsManager = accountsManager;
this.clock = clock;
}
static Pair<String, Long> getIdentifierAndDeviceId(final String basicUsername) {
final String identifier;
final long deviceId;
final int deviceIdSeparatorIndex = basicUsername.indexOf(DEVICE_ID_SEPARATOR);
if (deviceIdSeparatorIndex == -1) {
identifier = basicUsername;
deviceId = Device.MASTER_ID;
} else {
identifier = basicUsername.substring(0, deviceIdSeparatorIndex);
deviceId = Long.parseLong(basicUsername.substring(deviceIdSeparatorIndex + 1));
}
return new Pair<>(identifier, deviceId);
}
public Optional<AuthenticatedAccount> authenticate(BasicCredentials basicCredentials, boolean enabledRequired) {
boolean succeeded = false;
String failureReason = null;
try {
final UUID accountUuid;
final long deviceId;
{
final Pair<String, Long> identifierAndDeviceId = getIdentifierAndDeviceId(basicCredentials.getUsername());
accountUuid = UUID.fromString(identifierAndDeviceId.first());
deviceId = identifierAndDeviceId.second();
}
Optional<Account> account = accountsManager.getByAccountIdentifier(accountUuid);
if (account.isEmpty()) {
failureReason = "noSuchAccount";
return Optional.empty();
}
Optional<Device> device = account.get().getDevice(deviceId);
if (device.isEmpty()) {
failureReason = "noSuchDevice";
return Optional.empty();
}
if (enabledRequired) {
final boolean deviceDisabled = !device.get().isEnabled();
if (deviceDisabled) {
failureReason = "deviceDisabled";
}
final boolean accountDisabled = !account.get().isEnabled();
if (accountDisabled) {
failureReason = "accountDisabled";
}
if (accountDisabled || deviceDisabled) {
return Optional.empty();
}
} else {
Metrics.counter(ENABLED_NOT_REQUIRED_AUTHENTICATION_COUNTER_NAME,
ENABLED_TAG_NAME, String.valueOf(device.get().isEnabled() && account.get().isEnabled()),
IS_PRIMARY_DEVICE_TAG, String.valueOf(device.get().isMaster()))
.increment();
}
SaltedTokenHash deviceSaltedTokenHash = device.get().getAuthTokenHash();
if (deviceSaltedTokenHash.verify(basicCredentials.getPassword())) {
succeeded = true;
Account authenticatedAccount = updateLastSeen(account.get(), device.get());
if (deviceSaltedTokenHash.getVersion() != SaltedTokenHash.CURRENT_VERSION) {
authenticatedAccount = accountsManager.updateDeviceAuthentication(
authenticatedAccount,
device.get(),
SaltedTokenHash.generateFor(basicCredentials.getPassword())); // new credentials have current version
}
return Optional.of(new AuthenticatedAccount(
new RefreshingAccountAndDeviceSupplier(authenticatedAccount, device.get().getId(), accountsManager)));
}
return Optional.empty();
} catch (IllegalArgumentException | InvalidAuthorizationHeaderException iae) {
failureReason = "invalidHeader";
return Optional.empty();
} finally {
Tags tags = Tags.of(
AUTHENTICATION_SUCCEEDED_TAG_NAME, String.valueOf(succeeded));
if (StringUtils.isNotBlank(failureReason)) {
tags = tags.and(AUTHENTICATION_FAILURE_REASON_TAG_NAME, failureReason);
}
Metrics.counter(AUTHENTICATION_COUNTER_NAME, tags).increment();
}
}
@VisibleForTesting
public Account updateLastSeen(Account account, Device device) {
// compute a non-negative integer between 0 and 86400.
long n = Util.ensureNonNegativeLong(account.getUuid().getLeastSignificantBits());
final long lastSeenOffsetSeconds = n % ChronoUnit.DAYS.getDuration().toSeconds();
// produce a truncated timestamp which is either today at UTC midnight
// or yesterday at UTC midnight, based on per-user randomized offset used.
final long todayInMillisWithOffset = Util.todayInMillisGivenOffsetFromNow(clock, Duration.ofSeconds(lastSeenOffsetSeconds).negated());
// only update the device's last seen time when it falls behind the truncated timestamp.
// this ensure a few things:
// (1) each account will only update last-seen at most once per day
// (2) these updates will occur throughout the day rather than all occurring at UTC midnight.
if (device.getLastSeen() < todayInMillisWithOffset) {
Metrics.summary(DAYS_SINCE_LAST_SEEN_DISTRIBUTION_NAME, IS_PRIMARY_DEVICE_TAG, String.valueOf(device.isMaster()))
.record(Duration.ofMillis(todayInMillisWithOffset - device.getLastSeen()).toDays());
return accountsManager.updateDeviceLastSeen(account, device, Util.todayInMillis(clock));
}
return account;
}
}

View File

@@ -11,10 +11,10 @@ import org.whispersystems.textsecuregcm.util.Pair;
public class BasicAuthorizationHeader {
private final String username;
private final long deviceId;
private final byte deviceId;
private final String password;
private BasicAuthorizationHeader(final String username, final long deviceId, final String password) {
private BasicAuthorizationHeader(final String username, final byte deviceId, final String password) {
this.username = username;
this.deviceId = deviceId;
this.password = password;
@@ -59,10 +59,10 @@ public class BasicAuthorizationHeader {
final String usernameComponent = credentials.substring(0, credentialSeparatorIndex);
final String username;
final long deviceId;
final byte deviceId;
{
final Pair<String, Long> identifierAndDeviceId =
BaseAccountAuthenticator.getIdentifierAndDeviceId(usernameComponent);
final Pair<String, Byte> identifierAndDeviceId =
AccountAuthenticator.getIdentifierAndDeviceId(usernameComponent);
username = identifierAndDeviceId.first();
deviceId = identifierAndDeviceId.second();

View File

@@ -9,10 +9,10 @@ import com.google.protobuf.ByteString;
import com.google.protobuf.InvalidProtocolBufferException;
import java.security.InvalidKeyException;
import java.util.concurrent.TimeUnit;
import org.signal.libsignal.protocol.ecc.Curve;
import org.signal.libsignal.protocol.ecc.ECPrivateKey;
import org.whispersystems.textsecuregcm.entities.MessageProtos.SenderCertificate;
import org.whispersystems.textsecuregcm.entities.MessageProtos.ServerCertificate;
import org.whispersystems.textsecuregcm.identity.IdentityType;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
@@ -30,13 +30,13 @@ public class CertificateGenerator {
this.serverCertificate = ServerCertificate.parseFrom(serverCertificate);
}
public byte[] createFor(Account account, Device device, boolean includeE164) throws InvalidKeyException {
public byte[] createFor(final Account account, final byte deviceId, boolean includeE164) throws InvalidKeyException {
SenderCertificate.Certificate.Builder builder = SenderCertificate.Certificate.newBuilder()
.setSenderDevice(Math.toIntExact(device.getId()))
.setExpires(System.currentTimeMillis() + TimeUnit.DAYS.toMillis(expiresDays))
.setIdentityKey(ByteString.copyFrom(account.getIdentityKey().serialize()))
.setSigner(serverCertificate)
.setSenderUuid(account.getUuid().toString());
.setSenderDevice(Math.toIntExact(deviceId))
.setExpires(System.currentTimeMillis() + TimeUnit.DAYS.toMillis(expiresDays))
.setIdentityKey(ByteString.copyFrom(account.getIdentityKey(IdentityType.ACI).serialize()))
.setSigner(serverCertificate)
.setSenderUuid(account.getUuid().toString());
if (includeE164) {
builder.setSender(account.getNumber());
@@ -44,11 +44,7 @@ public class CertificateGenerator {
byte[] certificate = builder.build().toByteArray();
byte[] signature;
try {
signature = Curve.calculateSignature(privateKey, certificate);
} catch (org.signal.libsignal.protocol.InvalidKeyException e) {
throw new InvalidKeyException(e);
}
signature = privateKey.calculateSignature(certificate);
return SenderCertificate.newBuilder()
.setCertificate(ByteString.copyFrom(certificate))

View File

@@ -16,5 +16,5 @@ import java.lang.annotation.Target;
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface ChangesDeviceEnabledState {
public @interface ChangesLinkedDevices {
}

View File

@@ -0,0 +1,20 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* Indicates that an endpoint changes the phone number and PNI keys associated with an account, and that
* any websockets associated with the account may need to be refreshed after a call to that endpoint.
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface ChangesPhoneNumber {
}

View File

@@ -0,0 +1,137 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import com.fasterxml.jackson.core.JsonProcessingException;
import io.netty.resolver.dns.DnsNameResolver;
import jakarta.ws.rs.core.Response;
import java.io.IOException;
import java.net.Inet6Address;
import java.net.URI;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CompletionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ScheduledExecutorService;
import javax.annotation.Nullable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.http.FaultTolerantHttpClient;
import org.whispersystems.textsecuregcm.util.ExceptionUtils;
import org.whispersystems.textsecuregcm.util.SystemMapper;
public class CloudflareTurnCredentialsManager {
private static final Logger logger = LoggerFactory.getLogger(CloudflareTurnCredentialsManager.class);
private final List<String> cloudflareTurnUrls;
private final List<String> cloudflareTurnUrlsWithIps;
private final String cloudflareTurnHostname;
private final HttpRequest getCredentialsRequest;
private final FaultTolerantHttpClient cloudflareTurnClient;
private final DnsNameResolver dnsNameResolver;
private final Duration clientCredentialTtl;
private record CredentialRequest(long ttl) {}
private record CloudflareTurnResponse(IceServer iceServers) {
private record IceServer(
String username,
String credential,
List<String> urls) {
}
}
public CloudflareTurnCredentialsManager(final String cloudflareTurnApiToken,
final String cloudflareTurnEndpoint,
final Duration requestedCredentialTtl,
final Duration clientCredentialTtl,
final List<String> cloudflareTurnUrls,
final List<String> cloudflareTurnUrlsWithIps,
final String cloudflareTurnHostname,
final int cloudflareTurnNumHttpClients,
@Nullable final String circuitBreakerConfigurationName,
final ExecutorService executor,
@Nullable final String retryConfigurationName,
final ScheduledExecutorService retryExecutor,
final DnsNameResolver dnsNameResolver) {
this.cloudflareTurnClient = FaultTolerantHttpClient.newBuilder("cloudflare-turn", executor)
.withCircuitBreaker(circuitBreakerConfigurationName)
.withRetry(retryConfigurationName, retryExecutor)
.withNumClients(cloudflareTurnNumHttpClients)
.build();
this.cloudflareTurnUrls = cloudflareTurnUrls;
this.cloudflareTurnUrlsWithIps = cloudflareTurnUrlsWithIps;
this.cloudflareTurnHostname = cloudflareTurnHostname;
this.dnsNameResolver = dnsNameResolver;
final String credentialsRequestBody;
try {
credentialsRequestBody =
SystemMapper.jsonMapper().writeValueAsString(new CredentialRequest(requestedCredentialTtl.toSeconds()));
} catch (final JsonProcessingException e) {
throw new IllegalArgumentException(e);
}
// We repeat the same request to Cloudflare every time, so we can construct it once and re-use it
this.getCredentialsRequest = HttpRequest.newBuilder()
.uri(URI.create(cloudflareTurnEndpoint))
.header("Content-Type", "application/json")
.header("Authorization", String.format("Bearer %s", cloudflareTurnApiToken))
.POST(HttpRequest.BodyPublishers.ofString(credentialsRequestBody))
.build();
this.clientCredentialTtl = clientCredentialTtl;
}
public TurnToken retrieveFromCloudflare() throws IOException {
final List<String> cloudflareTurnComposedUrls;
try {
cloudflareTurnComposedUrls = dnsNameResolver.resolveAll(cloudflareTurnHostname).get().stream()
.map(i -> switch (i) {
case Inet6Address i6 -> "[" + i6.getHostAddress() + "]";
default -> i.getHostAddress();
})
.flatMap(i -> cloudflareTurnUrlsWithIps.stream().map(u -> u.formatted(i)))
.toList();
} catch (Exception e) {
throw new IOException(e);
}
final HttpResponse<String> response;
try {
response = cloudflareTurnClient.sendAsync(getCredentialsRequest, HttpResponse.BodyHandlers.ofString()).join();
} catch (CompletionException e) {
logger.warn("failed to make http request to Cloudflare Turn: {}", e.getMessage());
throw new IOException(ExceptionUtils.unwrap(e));
}
if (response.statusCode() != Response.Status.CREATED.getStatusCode()) {
logger.warn("failure request credentials from Cloudflare Turn (code={}): {}", response.statusCode(), response);
throw new IOException("Cloudflare Turn http failure : " + response.statusCode());
}
final CloudflareTurnResponse cloudflareTurnResponse = SystemMapper.jsonMapper()
.readValue(response.body(), CloudflareTurnResponse.class);
return new TurnToken(
cloudflareTurnResponse.iceServers().username(),
cloudflareTurnResponse.iceServers().credential(),
clientCredentialTtl.toSeconds(),
cloudflareTurnUrls == null ? Collections.emptyList() : cloudflareTurnUrls,
cloudflareTurnComposedUrls,
cloudflareTurnHostname
);
}
}

View File

@@ -5,10 +5,10 @@
package org.whispersystems.textsecuregcm.auth;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response;
import jakarta.ws.rs.core.Response.Status;
import java.util.Base64;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.Status;
public class CombinedUnidentifiedSenderAccessKeys {
private final byte[] combinedUnidentifiedSenderAccessKeys;
@@ -16,7 +16,7 @@ public class CombinedUnidentifiedSenderAccessKeys {
public CombinedUnidentifiedSenderAccessKeys(String header) {
try {
this.combinedUnidentifiedSenderAccessKeys = Base64.getDecoder().decode(header);
if (this.combinedUnidentifiedSenderAccessKeys == null || this.combinedUnidentifiedSenderAccessKeys.length != 16) {
if (this.combinedUnidentifiedSenderAccessKeys == null || this.combinedUnidentifiedSenderAccessKeys.length != UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH) {
throw new WebApplicationException("Invalid combined unidentified sender access keys", Status.UNAUTHORIZED);
}
} catch (IllegalArgumentException e) {

View File

@@ -1,21 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import org.glassfish.jersey.server.ContainerRequest;
import org.whispersystems.textsecuregcm.storage.Account;
import javax.ws.rs.core.SecurityContext;
import java.util.Optional;
class ContainerRequestUtil {
static Optional<Account> getAuthenticatedAccount(final ContainerRequest request) {
return Optional.ofNullable(request.getSecurityContext())
.map(SecurityContext::getUserPrincipal)
.map(principal -> principal instanceof AccountAndAuthenticatedDeviceHolder
? ((AccountAndAuthenticatedDeviceHolder) principal).getAccount() : null);
}
}

View File

@@ -1,26 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import io.dropwizard.auth.Authenticator;
import io.dropwizard.auth.basic.BasicCredentials;
import java.util.Optional;
import org.whispersystems.textsecuregcm.experiment.ExperimentEnrollmentManager;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
public class DisabledPermittedAccountAuthenticator extends BaseAccountAuthenticator implements
Authenticator<BasicCredentials, DisabledPermittedAuthenticatedAccount> {
public DisabledPermittedAccountAuthenticator(AccountsManager accountsManager) {
super(accountsManager);
}
@Override
public Optional<DisabledPermittedAuthenticatedAccount> authenticate(BasicCredentials credentials) {
Optional<AuthenticatedAccount> account = super.authenticate(credentials, false);
return account.map(DisabledPermittedAuthenticatedAccount::new);
}
}

View File

@@ -1,42 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.security.Principal;
import javax.security.auth.Subject;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
public class DisabledPermittedAuthenticatedAccount implements Principal, AccountAndAuthenticatedDeviceHolder {
private final AuthenticatedAccount authenticatedAccount;
public DisabledPermittedAuthenticatedAccount(final AuthenticatedAccount authenticatedAccount) {
this.authenticatedAccount = authenticatedAccount;
}
@Override
public Account getAccount() {
return authenticatedAccount.getAccount();
}
@Override
public Device getAuthenticatedDevice() {
return authenticatedAccount.getAuthenticatedDevice();
}
// Principal implementation
@Override
public String getName() {
return null;
}
@Override
public boolean implies(Subject subject) {
return false;
}
}

View File

@@ -0,0 +1,19 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
/**
* A disconnection request listener receives and handles a request to close an authenticated network connection for a
* specific client.
*/
public interface DisconnectionRequestListener {
/**
* Handles a request to close an authenticated network connection for a specific authenticated device. Requests are
* dispatched on dedicated threads, and implementations may safely block.
*/
void handleDisconnectionRequest();
}

View File

@@ -0,0 +1,231 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import com.google.common.annotations.VisibleForTesting;
import com.google.protobuf.InvalidProtocolBufferException;
import io.dropwizard.lifecycle.Managed;
import io.lettuce.core.RedisCommandTimeoutException;
import io.lettuce.core.pubsub.RedisPubSubAdapter;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Metrics;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Executor;
import java.util.concurrent.ScheduledExecutorService;
import javax.annotation.Nullable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.identity.IdentityType;
import org.whispersystems.textsecuregcm.metrics.MetricsUtil;
import org.whispersystems.textsecuregcm.redis.FaultTolerantPubSubConnection;
import org.whispersystems.textsecuregcm.redis.FaultTolerantRedisClient;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.util.ResilienceUtil;
import org.whispersystems.textsecuregcm.util.UUIDUtil;
/**
* A disconnection request manager broadcasts and dispatches requests for servers to close authenticated connections
* from specific clients.
*
* @see DisconnectionRequestListener
*/
public class DisconnectionRequestManager extends RedisPubSubAdapter<byte[], byte[]> implements Managed {
private final FaultTolerantRedisClient pubSubClient;
private final Executor listenerEventExecutor;
private final ScheduledExecutorService retryExecutor;
private static final String RETRY_NAME = ResilienceUtil.name(DisconnectionRequestManager.class);
private static final Duration SUBSCRIBE_RETRY_DELAY = Duration.ofSeconds(5);
private final Map<AccountIdentifierAndDeviceId, List<DisconnectionRequestListener>> listeners =
new ConcurrentHashMap<>();
@Nullable
private FaultTolerantPubSubConnection<byte[], byte[]> pubSubConnection;
private static final byte[] DISCONNECTION_REQUEST_CHANNEL = "disconnection_requests".getBytes(StandardCharsets.UTF_8);
private static final Counter DISCONNECTION_REQUESTS_SENT_COUNTER =
Metrics.counter(MetricsUtil.name(DisconnectionRequestManager.class, "requestsSent"));
private static final Counter DISCONNECTION_REQUESTS_RECEIVED_COUNTER =
Metrics.counter(MetricsUtil.name(DisconnectionRequestManager.class, "requestsReceived"));
private static final Logger logger = LoggerFactory.getLogger(DisconnectionRequestManager.class);
private record AccountIdentifierAndDeviceId(UUID accountIdentifier, byte deviceId) {}
public DisconnectionRequestManager(final FaultTolerantRedisClient pubSubClient,
final Executor listenerEventExecutor,
final ScheduledExecutorService retryExecutor) {
this.pubSubClient = pubSubClient;
this.listenerEventExecutor = listenerEventExecutor;
this.retryExecutor = retryExecutor;
}
@Override
public synchronized void start() {
this.pubSubConnection = pubSubClient.createBinaryPubSubConnection();
this.pubSubConnection.usePubSubConnection(connection -> {
connection.addListener(this);
boolean subscribed = false;
// Loop indefinitely until we establish a subscription. We don't want to fail immediately if there's a temporary
// Redis connectivity issue, since that would derail the whole startup process and likely lead to unnecessary pod
// churn, which might make things worse. If we never establish a connection, readiness probes will eventually fail
// and terminate the pods.
do {
try {
connection.sync().subscribe(DISCONNECTION_REQUEST_CHANNEL);
subscribed = true;
} catch (final RedisCommandTimeoutException e) {
try {
Thread.sleep(SUBSCRIBE_RETRY_DELAY);
} catch (final InterruptedException ex) {
throw new RuntimeException(ex);
}
}
} while (!subscribed);
});
}
@Override
public synchronized void stop() {
if (pubSubConnection != null) {
pubSubConnection.usePubSubConnection(connection -> {
connection.removeListener(this);
connection.close();
});
}
pubSubConnection = null;
}
/**
* Adds a listener for disconnection requests for a specific authenticated device.
*
* @param accountIdentifier TODO
* @param deviceId TODO
* @param listener the listener to register
*/
public void addListener(final UUID accountIdentifier, final byte deviceId, final DisconnectionRequestListener listener) {
listeners.compute(new AccountIdentifierAndDeviceId(accountIdentifier, deviceId), (_, existingListeners) -> {
final List<DisconnectionRequestListener> listeners =
existingListeners == null ? new ArrayList<>() : existingListeners;
listeners.add(listener);
return listeners;
});
}
/**
* Removes a listener for disconnection requests for a specific authenticated device.
*
* @param accountIdentifier TODO
* @param deviceId TODO
* @param listener the listener to remove
*/
public void removeListener(final UUID accountIdentifier, final byte deviceId, final DisconnectionRequestListener listener) {
listeners.computeIfPresent(new AccountIdentifierAndDeviceId(accountIdentifier, deviceId), (_, existingListeners) -> {
existingListeners.remove(listener);
return existingListeners.isEmpty() ? null : existingListeners;
});
}
@VisibleForTesting
List<DisconnectionRequestListener> getListeners(final UUID accountIdentifier, final byte deviceId) {
return listeners.getOrDefault(new AccountIdentifierAndDeviceId(accountIdentifier, deviceId), Collections.emptyList());
}
/**
* Broadcasts a request to close all connections associated with the given account identifier to all servers.
*
* @param account the account for which to close connections
*
* @return a future that completes when the request has been broadcast
*/
public CompletionStage<Void> requestDisconnection(final Account account) {
return requestDisconnection(account.getIdentifier(IdentityType.ACI),
account.getDevices().stream().map(Device::getId).toList());
}
/**
* Broadcasts a request to close connections associated with the given account identifier and device IDs to all
* servers.
*
* @param accountIdentifier the account for which to close connections
* @param deviceIds the device IDs for which to close connections
*
* @return a future that completes when the request has been broadcast
*/
public CompletionStage<Void> requestDisconnection(final UUID accountIdentifier, final Collection<Byte> deviceIds) {
final DisconnectionRequest disconnectionRequest = DisconnectionRequest.newBuilder()
.setAccountIdentifier(UUIDUtil.toByteString(accountIdentifier))
.addAllDeviceIds(deviceIds.stream().mapToInt(Byte::intValue).boxed().toList())
.build();
return ResilienceUtil.getGeneralRedisRetry(RETRY_NAME)
.executeCompletionStage(retryExecutor, () -> pubSubClient.withBinaryConnection(connection ->
connection.async().publish(DISCONNECTION_REQUEST_CHANNEL, disconnectionRequest.toByteArray()))
.toCompletableFuture())
.thenRun(DISCONNECTION_REQUESTS_SENT_COUNTER::increment);
}
@Override
public void message(final byte[] channel, final byte[] message) {
final UUID accountIdentifier;
final List<Byte> deviceIds;
try {
final DisconnectionRequest disconnectionRequest = DisconnectionRequest.parseFrom(message);
DISCONNECTION_REQUESTS_RECEIVED_COUNTER.increment();
accountIdentifier = UUIDUtil.fromByteString(disconnectionRequest.getAccountIdentifier());
deviceIds = disconnectionRequest.getDeviceIdsList().stream()
.map(deviceIdInt -> {
if (deviceIdInt == null || deviceIdInt < Device.PRIMARY_ID || deviceIdInt > Byte.MAX_VALUE) {
throw new IllegalArgumentException("Invalid device ID: " + deviceIdInt);
}
return deviceIdInt.byteValue();
})
.toList();
} catch (final InvalidProtocolBufferException e) {
logger.error("Could not parse disconnection request protobuf", e);
return;
} catch (final IllegalArgumentException e) {
logger.error("Could not parse part of disconnection request", e);
return;
}
deviceIds.forEach(deviceId -> {
listeners.getOrDefault(new AccountIdentifierAndDeviceId(accountIdentifier, deviceId), Collections.emptyList())
.forEach(listener -> listenerEventExecutor.execute(() -> {
try {
listener.handleDisconnectionRequest();
} catch (final Exception e) {
logger.warn("Listener failed to handle disconnection request", e);
}
}));
});
}
}

View File

@@ -154,7 +154,7 @@ public class ExternalServiceCredentialsGenerator {
/**
* Given an instance of {@link ExternalServiceCredentials} object, checks that the password
* matches the username taking into accound this generator's configuration.
* matches the username taking into account this generator's configuration.
* @param credentials an instance of {@link ExternalServiceCredentials}
* @return An optional with a timestamp (seconds) of when the credentials were generated,
* or an empty optional if the password doesn't match the username for any reason (including malformed data)

View File

@@ -0,0 +1,25 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response.Status;
import java.util.Base64;
import org.signal.libsignal.zkgroup.InvalidInputException;
import org.signal.libsignal.zkgroup.groupsend.GroupSendFullToken;
public record GroupSendTokenHeader(GroupSendFullToken token) {
public static GroupSendTokenHeader valueOf(String header) {
try {
return new GroupSendTokenHeader(new GroupSendFullToken(Base64.getDecoder().decode(header)));
} catch (InvalidInputException | IllegalArgumentException e) {
// Base64 throws IllegalArgumentException; GroupSendFullToken ctor throws InvalidInputException
throw new WebApplicationException(e, Status.UNAUTHORIZED);
}
}
}

View File

@@ -0,0 +1,58 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import com.google.common.annotations.VisibleForTesting;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Metrics;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.util.Optional;
import org.eclipse.jetty.websocket.server.JettyServerUpgradeRequest;
import org.eclipse.jetty.websocket.server.JettyServerUpgradeResponse;
import org.whispersystems.textsecuregcm.metrics.MetricsUtil;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.websocket.auth.AuthenticatedWebSocketUpgradeFilter;
public class IdlePrimaryDeviceAuthenticatedWebSocketUpgradeFilter implements
AuthenticatedWebSocketUpgradeFilter<AuthenticatedDevice> {
private final Duration minIdleDuration;
private final Clock clock;
@VisibleForTesting
static final String ALERT_HEADER = "X-Signal-Alert";
@VisibleForTesting
static final String IDLE_PRIMARY_DEVICE_ALERT = "idle-primary-device";
private static final Counter IDLE_PRIMARY_WARNING_COUNTER = Metrics.counter(
MetricsUtil.name(IdlePrimaryDeviceAuthenticatedWebSocketUpgradeFilter.class, "idlePrimaryDeviceWarning"),
"critical", "false");
public IdlePrimaryDeviceAuthenticatedWebSocketUpgradeFilter(final Duration minIdleDuration, final Clock clock) {
this.minIdleDuration = minIdleDuration;
this.clock = clock;
}
@Override
public void handleAuthentication(final Optional<AuthenticatedDevice> authenticated,
final JettyServerUpgradeRequest request,
final JettyServerUpgradeResponse response) {
// No action needed if the connection is unauthenticated (in which case we don't know when we've last seen the
// primary device) or if the authenticated device IS the primary device
authenticated
.filter(authenticatedDevice -> authenticatedDevice.deviceId() != Device.PRIMARY_ID)
.ifPresent(authenticatedDevice -> {
if (authenticatedDevice.primaryDeviceLastSeen().isBefore(clock.instant().minus(minIdleDuration))) {
response.addHeader(ALERT_HEADER, IDLE_PRIMARY_DEVICE_ALERT);
IDLE_PRIMARY_WARNING_COUNTER.increment();
}
});
}
}

View File

@@ -5,8 +5,8 @@
package org.whispersystems.textsecuregcm.auth;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response.Status;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response.Status;
public class InvalidAuthorizationHeaderException extends WebApplicationException {
public InvalidAuthorizationHeaderException(String s) {

View File

@@ -5,35 +5,37 @@
package org.whispersystems.textsecuregcm.auth;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
import javax.ws.rs.NotAuthorizedException;
import javax.ws.rs.NotFoundException;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
import jakarta.ws.rs.NotAuthorizedException;
import jakarta.ws.rs.NotFoundException;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response;
import java.security.MessageDigest;
import java.util.Optional;
import org.whispersystems.textsecuregcm.identity.IdentityType;
import org.whispersystems.textsecuregcm.identity.ServiceIdentifier;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.Device;
@SuppressWarnings("OptionalUsedAsFieldOrParameterType")
public class OptionalAccess {
public static final String UNIDENTIFIED = "Unidentified-Access-Key";
public static String ALL_DEVICES_SELECTOR = "*";
public static void verify(Optional<Account> requestAccount,
Optional<Anonymous> accessKey,
Optional<Account> targetAccount,
ServiceIdentifier targetIdentifier,
String deviceSelector) {
public static void verify(Optional<Account> requestAccount,
Optional<Anonymous> accessKey,
Optional<Account> targetAccount,
String deviceSelector)
{
try {
verify(requestAccount, accessKey, targetAccount);
verify(requestAccount, accessKey, targetAccount, targetIdentifier);
if (!deviceSelector.equals("*")) {
long deviceId = Long.parseLong(deviceSelector);
if (!ALL_DEVICES_SELECTOR.equals(deviceSelector)) {
byte deviceId = Byte.parseByte(deviceSelector);
Optional<Device> targetDevice = targetAccount.get().getDevice(deviceId);
if (targetDevice.isPresent() && targetDevice.get().isEnabled()) {
if (targetDevice.isPresent()) {
return;
}
@@ -48,29 +50,50 @@ public class OptionalAccess {
}
}
public static void verify(Optional<Account> requestAccount,
Optional<Anonymous> accessKey,
Optional<Account> targetAccount)
{
if (requestAccount.isPresent() && targetAccount.isPresent() && targetAccount.get().isEnabled()) {
public static void verify(Optional<Account> requestAccount,
Optional<Anonymous> accessKey,
Optional<Account> targetAccount,
ServiceIdentifier targetIdentifier) {
if (requestAccount.isPresent()) {
// Authenticated requests are never unauthorized; if the target exists, return OK, otherwise throw not-found.
if (targetAccount.isPresent()) {
return;
} else {
throw new NotFoundException();
}
}
// Anything past this point can only be authenticated by an access key. Even when the target
// has unrestricted unidentified access, callers need to supply a fake access key. Likewise, if
// the target account does not exist, we *also* report unauthorized here (*not* not-found,
// since that would provide a free exists check).
if (accessKey.isEmpty() || targetAccount.isEmpty()) {
throw new NotAuthorizedException(Response.Status.UNAUTHORIZED);
}
// Unidentified access is only for ACI identities
if (IdentityType.PNI.equals(targetIdentifier.identityType())) {
throw new NotAuthorizedException(Response.Status.UNAUTHORIZED);
}
// Unrestricted unidentified access does what it says on the tin: we don't check if the key the
// caller provided is right or not.
if (targetAccount.get().isUnrestrictedUnidentifiedAccess()) {
return;
}
//noinspection ConstantConditions
if (requestAccount.isPresent() && (targetAccount.isEmpty() || (targetAccount.isPresent() && !targetAccount.get().isEnabled()))) {
throw new NotFoundException();
if (!targetAccount.get().isIdentifiedBy(targetIdentifier)) {
throw new IllegalArgumentException("Target account is not identified by the given identifier");
}
if (accessKey.isPresent() && targetAccount.isPresent() && targetAccount.get().isEnabled() && targetAccount.get().isUnrestrictedUnidentifiedAccess()) {
return;
// At this point, any successful authentication requires a real access key on the target account
if (targetAccount.get().getUnidentifiedAccessKey().isEmpty()) {
throw new NotAuthorizedException(Response.Status.UNAUTHORIZED);
}
if (accessKey.isPresent() &&
targetAccount.isPresent() &&
targetAccount.get().getUnidentifiedAccessKey().isPresent() &&
targetAccount.get().isEnabled() &&
MessageDigest.isEqual(accessKey.get().getAccessKey(), targetAccount.get().getUnidentifiedAccessKey().get()))
{
// Otherwise, access is gated by the caller having the unidentified-access key matching the target account.
if (MessageDigest.isEqual(accessKey.get().getAccessKey(), targetAccount.get().getUnidentifiedAccessKey().get())) {
return;
}

View File

@@ -1,46 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
import java.util.stream.Collectors;
import org.glassfish.jersey.server.monitoring.RequestEvent;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.util.Pair;
public class PhoneNumberChangeRefreshRequirementProvider implements WebsocketRefreshRequirementProvider {
private static final String INITIAL_NUMBER_KEY =
PhoneNumberChangeRefreshRequirementProvider.class.getName() + ".initialNumber";
@Override
public void handleRequestFiltered(final RequestEvent requestEvent) {
ContainerRequestUtil.getAuthenticatedAccount(requestEvent.getContainerRequest())
.ifPresent(account -> requestEvent.getContainerRequest().setProperty(INITIAL_NUMBER_KEY, account.getNumber()));
}
@Override
public List<Pair<UUID, Long>> handleRequestFinished(final RequestEvent requestEvent) {
final String initialNumber = (String) requestEvent.getContainerRequest().getProperty(INITIAL_NUMBER_KEY);
if (initialNumber != null) {
final Optional<Account> maybeAuthenticatedAccount =
ContainerRequestUtil.getAuthenticatedAccount(requestEvent.getContainerRequest());
return maybeAuthenticatedAccount
.filter(account -> !initialNumber.equals(account.getNumber()))
.map(account -> account.getDevices().stream()
.map(device -> new Pair<>(account.getUuid(), device.getId()))
.collect(Collectors.toList()))
.orElse(Collections.emptyList());
} else {
return Collections.emptyList();
}
}
}

View File

@@ -7,22 +7,25 @@ package org.whispersystems.textsecuregcm.auth;
import io.grpc.Status;
import io.grpc.StatusRuntimeException;
import jakarta.ws.rs.BadRequestException;
import jakarta.ws.rs.ForbiddenException;
import jakarta.ws.rs.NotAuthorizedException;
import jakarta.ws.rs.ServerErrorException;
import jakarta.ws.rs.container.ContainerRequestContext;
import jakarta.ws.rs.core.Response;
import java.security.MessageDigest;
import java.time.Duration;
import java.util.concurrent.CancellationException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import javax.ws.rs.BadRequestException;
import javax.ws.rs.ForbiddenException;
import javax.ws.rs.NotAuthorizedException;
import javax.ws.rs.ServerErrorException;
import javax.ws.rs.core.Response;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.entities.PhoneVerificationRequest;
import org.whispersystems.textsecuregcm.entities.RegistrationServiceSession;
import org.whispersystems.textsecuregcm.registration.RegistrationServiceClient;
import org.whispersystems.textsecuregcm.spam.RegistrationRecoveryChecker;
import org.whispersystems.textsecuregcm.storage.PhoneNumberIdentifiers;
import org.whispersystems.textsecuregcm.storage.RegistrationRecoveryPasswordsManager;
public class PhoneVerificationTokenManager {
@@ -31,19 +34,27 @@ public class PhoneVerificationTokenManager {
private static final Duration REGISTRATION_RPC_TIMEOUT = Duration.ofSeconds(15);
private static final long VERIFICATION_TIMEOUT_SECONDS = REGISTRATION_RPC_TIMEOUT.plusSeconds(1).getSeconds();
private final PhoneNumberIdentifiers phoneNumberIdentifiers;
private final RegistrationServiceClient registrationServiceClient;
private final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager;
private final RegistrationRecoveryChecker registrationRecoveryChecker;
public PhoneVerificationTokenManager(final RegistrationServiceClient registrationServiceClient,
final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager) {
public PhoneVerificationTokenManager(final PhoneNumberIdentifiers phoneNumberIdentifiers,
final RegistrationServiceClient registrationServiceClient,
final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager,
final RegistrationRecoveryChecker registrationRecoveryChecker) {
this.phoneNumberIdentifiers = phoneNumberIdentifiers;
this.registrationServiceClient = registrationServiceClient;
this.registrationRecoveryPasswordsManager = registrationRecoveryPasswordsManager;
this.registrationRecoveryChecker = registrationRecoveryChecker;
}
/**
* Checks if a {@link PhoneVerificationRequest} has a token that verifies the caller has confirmed access to the e164
* number
*
* @param requestContext the container request context
* @param number the e164 presented for verification
* @param request the request with exactly one verification token (RegistrationService sessionId or registration
* recovery password)
@@ -54,13 +65,13 @@ public class PhoneVerificationTokenManager {
* @throws ForbiddenException if the recovery password is not valid
* @throws InterruptedException if verification did not complete before a timeout
*/
public PhoneVerificationRequest.VerificationType verify(final String number, final PhoneVerificationRequest request)
public PhoneVerificationRequest.VerificationType verify(final ContainerRequestContext requestContext, final String number, final PhoneVerificationRequest request)
throws InterruptedException {
final PhoneVerificationRequest.VerificationType verificationType = request.verificationType();
switch (verificationType) {
case SESSION -> verifyBySessionId(number, request.decodeSessionId());
case RECOVERY_PASSWORD -> verifyByRecoveryPassword(number, request.recoveryPassword());
case RECOVERY_PASSWORD -> verifyByRecoveryPassword(requestContext, number, request.recoveryPassword());
}
return verificationType;
@@ -97,10 +108,13 @@ public class PhoneVerificationTokenManager {
}
}
private void verifyByRecoveryPassword(final String number, final byte[] recoveryPassword)
private void verifyByRecoveryPassword(final ContainerRequestContext requestContext, final String number, final byte[] recoveryPassword)
throws InterruptedException {
if (!registrationRecoveryChecker.checkRegistrationRecoveryAttempt(requestContext, number)) {
throw new ForbiddenException("recoveryPassword couldn't be verified");
}
try {
final boolean verified = registrationRecoveryPasswordsManager.verify(number, recoveryPassword)
final boolean verified = registrationRecoveryPasswordsManager.verify(phoneNumberIdentifiers.getPhoneNumberIdentifier(number).join(), recoveryPassword)
.get(VERIFICATION_TIMEOUT_SECONDS, TimeUnit.SECONDS);
if (!verified) {
throw new ForbiddenException("recoveryPassword couldn't be verified");

View File

@@ -0,0 +1,103 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneOffset;
import java.time.temporal.ChronoUnit;
import java.util.Iterator;
import java.util.Objects;
import java.util.stream.Stream;
import org.jetbrains.annotations.NotNull;
/// A validated range of days for which a credential may be issued.
public class RedemptionRange implements Iterable<Instant> {
public static final Duration MAX_REDEMPTION_DURATION = Duration.ofDays(7);
/// The first day for which a credential should be issued
private final LocalDate from;
/// The last day for which a credential should be issued
private final LocalDate end;
private RedemptionRange(final LocalDate from, final LocalDate end) {
this.from = from;
this.end = end;
}
/// Construct a {@link RedemptionRange} if the provided day bounds are valid.
///
/// The redemption bounds must satisfy:
/// - `redemptionEnd` >= `redemptionStart`
/// - `redemptionStart` and `redemptionEnd` are day-aligned
/// - `redemptionStart` is yesterday or later
/// - `redemptionEnd` is tomorrow + `MAX_REDEMPTION_DURATION` or earlier
/// - The number of days requested is less than `MAX_REDEMPTION_DURATION`
///
/// @param clock Clock to use to get current day
/// @param redemptionStart The first day included in the range
/// @param redemptionEnd The last day included in the range
/// @return A {@link RedemptionRange} that can be used to iterate each day between `redemptionStart` and
/// `redemptionEnd`
/// @throws IllegalArgumentException if the redemption bounds were not valid
public static RedemptionRange inclusive(Clock clock, Instant redemptionStart, Instant redemptionEnd)
throws IllegalArgumentException {
final Instant today = clock.instant().truncatedTo(ChronoUnit.DAYS);
final Instant yesterday = today.minus(Duration.ofDays(1));
if (redemptionStart.isAfter(redemptionEnd)) {
throw new IllegalArgumentException("end of range must be after start of range");
}
if (!redemptionStart.truncatedTo(ChronoUnit.DAYS).equals(redemptionStart)
|| !redemptionEnd.truncatedTo(ChronoUnit.DAYS).equals(redemptionEnd)) {
throw new IllegalArgumentException("timestamps must be day aligned");
}
if (redemptionStart.isBefore(yesterday)) {
throw new IllegalArgumentException("start of range too far in the past");
}
if (redemptionEnd.isAfter(today.plus(MAX_REDEMPTION_DURATION).plus(Duration.ofDays(1)))) {
throw new IllegalArgumentException("end of range too far in the future");
}
if (redemptionEnd.isAfter(redemptionStart.plus(MAX_REDEMPTION_DURATION))) {
throw new IllegalArgumentException("redemption window too large");
}
return new RedemptionRange(
LocalDate.ofInstant(redemptionStart, ZoneOffset.UTC),
LocalDate.ofInstant(redemptionEnd, ZoneOffset.UTC));
}
@Override
public @NotNull Iterator<Instant> iterator() {
final Instant fromInstant = from.atStartOfDay(ZoneOffset.UTC).toInstant();
final Instant endInstant = end.atStartOfDay(ZoneOffset.UTC).toInstant();
return Stream
.iterate(fromInstant, redemptionTime -> redemptionTime.plus(Duration.ofDays(1)))
.takeWhile(redemptionTime -> !redemptionTime.isAfter(endInstant))
.iterator();
}
@Override
public boolean equals(final Object o) {
if (o == null || getClass() != o.getClass()) {
return false;
}
RedemptionRange that = (RedemptionRange) o;
return Objects.equals(from, that.from) && Objects.equals(end, that.end);
}
@Override
public int hashCode() {
return Objects.hash(from, end);
}
}

View File

@@ -12,25 +12,25 @@ import io.micrometer.core.instrument.DistributionSummary;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.Tags;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import javax.annotation.Nullable;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
import org.apache.commons.lang3.StringUtils;
import org.whispersystems.textsecuregcm.controllers.RateLimitExceededException;
import org.whispersystems.textsecuregcm.entities.PhoneVerificationRequest;
import org.whispersystems.textsecuregcm.entities.RegistrationLockFailure;
import org.whispersystems.textsecuregcm.identity.IdentityType;
import org.whispersystems.textsecuregcm.limits.RateLimiters;
import org.whispersystems.textsecuregcm.metrics.UserAgentTagUtil;
import org.whispersystems.textsecuregcm.push.ClientPresenceManager;
import org.whispersystems.textsecuregcm.push.NotPushRegisteredException;
import org.whispersystems.textsecuregcm.push.PushNotificationManager;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.storage.RegistrationRecoveryPasswordsManager;
import org.whispersystems.textsecuregcm.util.Util;
public class RegistrationLockVerificationManager {
public enum Flow {
@@ -53,23 +53,21 @@ public class RegistrationLockVerificationManager {
private static final String PHONE_VERIFICATION_TYPE_TAG_NAME = "phoneVerificationType";
private final AccountsManager accounts;
private final ClientPresenceManager clientPresenceManager;
private final ExternalServiceCredentialsGenerator svr1CredentialGenerator;
private final DisconnectionRequestManager disconnectionRequestManager;
private final ExternalServiceCredentialsGenerator svr2CredentialGenerator;
private final RateLimiters rateLimiters;
private final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager;
private final PushNotificationManager pushNotificationManager;
public RegistrationLockVerificationManager(
final AccountsManager accounts, final ClientPresenceManager clientPresenceManager,
final ExternalServiceCredentialsGenerator svr1CredentialGenerator,
final AccountsManager accounts,
final DisconnectionRequestManager disconnectionRequestManager,
final ExternalServiceCredentialsGenerator svr2CredentialGenerator,
final RegistrationRecoveryPasswordsManager registrationRecoveryPasswordsManager,
final PushNotificationManager pushNotificationManager,
final RateLimiters rateLimiters) {
this.accounts = accounts;
this.clientPresenceManager = clientPresenceManager;
this.svr1CredentialGenerator = svr1CredentialGenerator;
this.disconnectionRequestManager = disconnectionRequestManager;
this.svr2CredentialGenerator = svr2CredentialGenerator;
this.registrationRecoveryPasswordsManager = registrationRecoveryPasswordsManager;
this.pushNotificationManager = pushNotificationManager;
@@ -109,7 +107,7 @@ public class RegistrationLockVerificationManager {
throw new RuntimeException("Unexpected status: " + existingRegistrationLock.getStatus());
}
if (!Util.isEmpty(clientRegistrationLock)) {
if (StringUtils.isNotEmpty(clientRegistrationLock)) {
rateLimiters.getPinLimiter().validate(account.getNumber());
}
@@ -141,9 +139,6 @@ public class RegistrationLockVerificationManager {
// Freezing the existing account credentials will definitively start the reglock timeout.
// Until the timeout, the current reglock can still be supplied,
// along with phone number verification, to restore access.
final ExternalServiceCredentials existingSvr1Credentials = svr1CredentialGenerator.generateForUuid(account.getUuid());
final ExternalServiceCredentials existingSvr2Credentials = svr2CredentialGenerator.generateForUuid(account.getUuid());
final Account updatedAccount;
if (!alreadyLocked) {
updatedAccount = accounts.update(account, Account::lockAuthTokenHash);
@@ -158,11 +153,11 @@ public class RegistrationLockVerificationManager {
// This allows users to re-register via registration recovery password
// instead of always being forced to fall back to SMS verification.
if (!phoneVerificationType.equals(PhoneVerificationRequest.VerificationType.RECOVERY_PASSWORD) || clientRegistrationLock != null) {
registrationRecoveryPasswordsManager.removeForNumber(updatedAccount.getNumber());
registrationRecoveryPasswordsManager.remove(updatedAccount.getIdentifier(IdentityType.PNI));
}
final List<Long> deviceIds = updatedAccount.getDevices().stream().map(Device::getId).toList();
clientPresenceManager.disconnectAllPresences(updatedAccount.getUuid(), deviceIds);
final List<Byte> deviceIds = updatedAccount.getDevices().stream().map(Device::getId).toList();
disconnectionRequestManager.requestDisconnection(updatedAccount.getUuid(), deviceIds);
try {
// Send a push notification that prompts the client to attempt login and fail due to locked credentials
@@ -172,12 +167,20 @@ public class RegistrationLockVerificationManager {
}
throw new WebApplicationException(Response.status(FAILURE_HTTP_STATUS)
.entity(new RegistrationLockFailure(existingRegistrationLock.getTimeRemaining().toMillis(),
existingRegistrationLock.needsFailureCredentials() ? existingSvr1Credentials : null,
existingRegistrationLock.needsFailureCredentials() ? existingSvr2Credentials : null))
.entity(new RegistrationLockFailure(
existingRegistrationLock.getTimeRemaining().toMillis(),
svr2FailureCredentials(existingRegistrationLock, updatedAccount)))
.build());
}
rateLimiters.getPinLimiter().clear(phoneNumber);
}
private @Nullable ExternalServiceCredentials svr2FailureCredentials(final StoredRegistrationLock existingRegistrationLock, final Account account) {
if (!existingRegistrationLock.needsFailureCredentials()) {
return null;
}
return svr2CredentialGenerator.generateForUuid(account.getUuid());
}
}

View File

@@ -11,7 +11,7 @@ import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.Optional;
import javax.annotation.Nullable;
import org.whispersystems.textsecuregcm.util.Util;
import org.apache.commons.lang3.StringUtils;
@SuppressWarnings("OptionalUsedAsFieldOrParameterType")
public class StoredRegistrationLock {
@@ -73,16 +73,11 @@ public class StoredRegistrationLock {
}
public boolean verify(@Nullable String clientRegistrationLock) {
if (hasLockAndSalt() && Util.nonEmpty(clientRegistrationLock)) {
if (hasLockAndSalt() && StringUtils.isNotEmpty(clientRegistrationLock)) {
SaltedTokenHash credentials = new SaltedTokenHash(registrationLock.get(), registrationLockSalt.get());
return credentials.verify(clientRegistrationLock);
} else {
return false;
}
}
@VisibleForTesting
public StoredRegistrationLock forTime(long timestamp) {
return new StoredRegistrationLock(registrationLock, registrationLockSalt, Instant.ofEpochMilli(timestamp));
}
}

View File

@@ -5,7 +5,16 @@
package org.whispersystems.textsecuregcm.auth;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.util.List;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;
public record TurnToken(String username, String password, List<String> urls) {
public record TurnToken(
String username,
String password,
@JsonProperty("ttl") long ttlSeconds,
@Nonnull List<String> urls,
@Nonnull List<String> urlsWithIps,
@Nullable String hostname) {
}

View File

@@ -1,78 +0,0 @@
/*
* Copyright 2013-2020 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import org.whispersystems.textsecuregcm.configuration.TurnUriConfiguration;
import org.whispersystems.textsecuregcm.configuration.dynamic.DynamicConfiguration;
import org.whispersystems.textsecuregcm.configuration.dynamic.DynamicTurnConfiguration;
import org.whispersystems.textsecuregcm.storage.DynamicConfigurationManager;
import org.whispersystems.textsecuregcm.util.Pair;
import org.whispersystems.textsecuregcm.util.Util;
import org.whispersystems.textsecuregcm.util.WeightedRandomSelect;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.time.Duration;
import java.time.Instant;
import java.util.Base64;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
public class TurnTokenGenerator {
private final DynamicConfigurationManager<DynamicConfiguration> dynamicConfigurationManager;
private final byte[] turnSecret;
private static final String ALGORITHM = "HmacSHA1";
public TurnTokenGenerator(final DynamicConfigurationManager<DynamicConfiguration> dynamicConfigurationManager,
final byte[] turnSecret) {
this.dynamicConfigurationManager = dynamicConfigurationManager;
this.turnSecret = turnSecret;
}
public TurnToken generate(final UUID aci) {
try {
final List<String> urls = urls(aci);
final Mac mac = Mac.getInstance(ALGORITHM);
final long validUntilSeconds = Instant.now().plus(Duration.ofDays(1)).getEpochSecond();
final long user = Util.ensureNonNegativeInt(new SecureRandom().nextInt());
final String userTime = validUntilSeconds + ":" + user;
mac.init(new SecretKeySpec(turnSecret, ALGORITHM));
final String password = Base64.getEncoder().encodeToString(mac.doFinal(userTime.getBytes()));
return new TurnToken(userTime, password, urls);
} catch (final NoSuchAlgorithmException | InvalidKeyException e) {
throw new AssertionError(e);
}
}
private List<String> urls(final UUID aci) {
final DynamicTurnConfiguration turnConfig = dynamicConfigurationManager.getConfiguration().getTurnConfiguration();
// Check if number is enrolled to test out specific turn servers
final Optional<TurnUriConfiguration> enrolled = turnConfig.getUriConfigs().stream()
.filter(config -> config.getEnrolledAcis().contains(aci))
.findFirst();
if (enrolled.isPresent()) {
return enrolled.get().getUris();
}
// Otherwise, select from turn server sets by weighted choice
return WeightedRandomSelect.select(turnConfig
.getUriConfigs()
.stream()
.map(c -> new Pair<>(c.getUris(), c.getWeight())).toList());
}
}

View File

@@ -9,23 +9,21 @@ import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.Base64;
import java.util.Optional;
@SuppressWarnings("OptionalUsedAsFieldOrParameterType")
public class UnidentifiedAccessChecksum {
public static String generateFor(Optional<byte[]> unidentifiedAccessKey) {
public static byte[] generateFor(byte[] unidentifiedAccessKey) {
try {
if (!unidentifiedAccessKey.isPresent()|| unidentifiedAccessKey.get().length != 16) return null;
if (unidentifiedAccessKey.length != UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH) {
throw new IllegalArgumentException("Invalid UAK length: " + unidentifiedAccessKey.length);
}
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(unidentifiedAccessKey.get(), "HmacSHA256"));
mac.init(new SecretKeySpec(unidentifiedAccessKey, "HmacSHA256"));
return Base64.getEncoder().encodeToString(mac.doFinal(new byte[32]));
return mac.doFinal(new byte[32]);
} catch (NoSuchAlgorithmException | InvalidKeyException e) {
throw new AssertionError(e);
}
}
}

View File

@@ -7,9 +7,14 @@ package org.whispersystems.textsecuregcm.auth;
import org.whispersystems.textsecuregcm.storage.Account;
import java.security.MessageDigest;
import java.util.Collection;
import java.util.function.Predicate;
import java.util.stream.IntStream;
public class UnidentifiedAccessUtil {
public static final int UNIDENTIFIED_ACCESS_KEY_LENGTH = 16;
private UnidentifiedAccessUtil() {
}
@@ -29,4 +34,42 @@ public class UnidentifiedAccessUtil {
.map(targetUnidentifiedAccessKey -> MessageDigest.isEqual(targetUnidentifiedAccessKey, unidentifiedAccessKey))
.orElse(false);
}
/**
* Checks whether an action (e.g. sending a message or retrieving pre-keys) may be taken on the collection of target
* accounts by an actor presenting the given combined unidentified access key.
*
* @param targetAccounts the accounts on which an actor wishes to take an action
* @param combinedUnidentifiedAccessKey the unidentified access key presented by the actor
*
* @return {@code true} if an actor presenting the given unidentified access key has permission to take an action on
* the target accounts or {@code false} otherwise
*/
public static boolean checkUnidentifiedAccess(final Collection<Account> targetAccounts, final byte[] combinedUnidentifiedAccessKey) {
return MessageDigest.isEqual(getCombinedUnidentifiedAccessKey(targetAccounts), combinedUnidentifiedAccessKey);
}
/**
* Calculates a combined unidentified access key for the given collection of accounts.
*
* @param accounts the accounts from which to derive a combined unidentified access key
* @return a combined unidentified access key
*
* @throws IllegalArgumentException if one or more of the given accounts had an unidentified access key with an
* unexpected length
*/
public static byte[] getCombinedUnidentifiedAccessKey(final Collection<Account> accounts) {
return accounts.stream()
.filter(Predicate.not(Account::isUnrestrictedUnidentifiedAccess))
.map(account ->
account.getUnidentifiedAccessKey()
.filter(b -> b.length == UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH)
.orElseThrow(IllegalArgumentException::new))
.reduce(new byte[UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH],
(a, b) -> {
final byte[] xor = new byte[UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH];
IntStream.range(0, UnidentifiedAccessUtil.UNIDENTIFIED_ACCESS_KEY_LENGTH).forEach(i -> xor[i] = (byte) (a[i] ^ b[i]));
return xor;
});
}
}

View File

@@ -1,38 +0,0 @@
/*
* Copyright 2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import org.glassfish.jersey.server.monitoring.ApplicationEvent;
import org.glassfish.jersey.server.monitoring.ApplicationEventListener;
import org.glassfish.jersey.server.monitoring.RequestEvent;
import org.glassfish.jersey.server.monitoring.RequestEventListener;
import org.whispersystems.textsecuregcm.push.ClientPresenceManager;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
/**
* Delegates request events to a listener that watches for intra-request changes that require websocket refreshes
*/
public class WebsocketRefreshApplicationEventListener implements ApplicationEventListener {
private final WebsocketRefreshRequestEventListener websocketRefreshRequestEventListener;
public WebsocketRefreshApplicationEventListener(final AccountsManager accountsManager,
final ClientPresenceManager clientPresenceManager) {
this.websocketRefreshRequestEventListener = new WebsocketRefreshRequestEventListener(clientPresenceManager,
new AuthEnablementRefreshRequirementProvider(accountsManager),
new PhoneNumberChangeRefreshRequirementProvider());
}
@Override
public void onEvent(final ApplicationEvent event) {
}
@Override
public RequestEventListener onRequest(final RequestEvent requestEvent) {
return websocketRefreshRequestEventListener;
}
}

View File

@@ -1,74 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import static org.whispersystems.textsecuregcm.metrics.MetricsUtil.name;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Metrics;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicInteger;
import javax.ws.rs.container.ResourceInfo;
import javax.ws.rs.core.Context;
import org.glassfish.jersey.server.monitoring.RequestEvent;
import org.glassfish.jersey.server.monitoring.RequestEvent.Type;
import org.glassfish.jersey.server.monitoring.RequestEventListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.push.ClientPresenceManager;
public class WebsocketRefreshRequestEventListener implements RequestEventListener {
private final ClientPresenceManager clientPresenceManager;
private final WebsocketRefreshRequirementProvider[] providers;
private static final Counter DISPLACED_ACCOUNTS = Metrics.counter(
name(WebsocketRefreshRequestEventListener.class, "displacedAccounts"));
private static final Counter DISPLACED_DEVICES = Metrics.counter(
name(WebsocketRefreshRequestEventListener.class, "displacedDevices"));
private static final Logger logger = LoggerFactory.getLogger(WebsocketRefreshRequestEventListener.class);
public WebsocketRefreshRequestEventListener(
final ClientPresenceManager clientPresenceManager,
final WebsocketRefreshRequirementProvider... providers) {
this.clientPresenceManager = clientPresenceManager;
this.providers = providers;
}
@Context
private ResourceInfo resourceInfo;
@Override
public void onEvent(final RequestEvent event) {
if (event.getType() == Type.REQUEST_FILTERED) {
for (final WebsocketRefreshRequirementProvider provider : providers) {
provider.handleRequestFiltered(event);
}
} else if (event.getType() == Type.FINISHED) {
final AtomicInteger displacedDevices = new AtomicInteger(0);
Arrays.stream(providers)
.flatMap(provider -> provider.handleRequestFinished(event).stream())
.distinct()
.forEach(pair -> {
try {
displacedDevices.incrementAndGet();
clientPresenceManager.disconnectPresence(pair.first(), pair.second());
} catch (final Exception e) {
logger.error("Could not displace device presence", e);
}
});
if (displacedDevices.get() > 0) {
DISPLACED_ACCOUNTS.increment();
DISPLACED_DEVICES.increment(displacedDevices.get());
}
}
}
}

View File

@@ -1,34 +0,0 @@
/*
* Copyright 2013-2021 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth;
import java.util.List;
import java.util.UUID;
import org.glassfish.jersey.server.monitoring.RequestEvent;
import org.whispersystems.textsecuregcm.util.Pair;
/**
* A websocket refresh requirement provider watches for intra-request changes (e.g. to authentication status) that
* require a websocket refresh.
*/
public interface WebsocketRefreshRequirementProvider {
/**
* Processes a request after filters have run and the request has been mapped to a destination controller.
*
* @param requestEvent the request event to observe
*/
void handleRequestFiltered(RequestEvent requestEvent);
/**
* Processes a request after all normal request handling has been completed.
*
* @param requestEvent the request event to observe
* @return a list of pairs of account UUID/device ID pairs identifying websockets that need to be refreshed as a
* result of the observed request
*/
List<Pair<UUID, Long>> handleRequestFinished(RequestEvent requestEvent);
}

View File

@@ -7,5 +7,5 @@ package org.whispersystems.textsecuregcm.auth.grpc;
import java.util.UUID;
public record AuthenticatedDevice(UUID accountIdentifier, long deviceId) {
public record AuthenticatedDevice(UUID accountIdentifier, byte deviceId) {
}

View File

@@ -7,37 +7,53 @@ package org.whispersystems.textsecuregcm.auth.grpc;
import io.grpc.Context;
import io.grpc.Status;
import java.util.UUID;
import javax.annotation.Nullable;
import reactor.core.publisher.Mono;
import reactor.util.function.Tuple2;
import reactor.util.function.Tuples;
import org.whispersystems.textsecuregcm.storage.Device;
/**
* Provides utility methods for working with authentication in the context of gRPC calls.
*/
public class AuthenticationUtil {
static final Context.Key<UUID> CONTEXT_AUTHENTICATED_ACCOUNT_IDENTIFIER_KEY = Context.key("authenticated-aci");
static final Context.Key<Long> CONTEXT_AUTHENTICATED_DEVICE_IDENTIFIER_KEY = Context.key("authenticated-device-id");
static final Context.Key<AuthenticatedDevice> CONTEXT_AUTHENTICATED_DEVICE = Context.key("authenticated-device");
/**
* Returns the account/device authenticated in the current gRPC context or throws an "unauthenticated" exception if
* no authenticated account/device is available.
*
* @return the account/device authenticated in the current gRPC context
* @return the account/device identifier authenticated in the current gRPC context
*
* @throws io.grpc.StatusRuntimeException with a status of {@code UNAUTHENTICATED} if no authenticated account/device
* could be retrieved from the current gRPC context
*/
public static AuthenticatedDevice requireAuthenticatedDevice() {
@Nullable final UUID accountIdentifier = CONTEXT_AUTHENTICATED_ACCOUNT_IDENTIFIER_KEY.get();
@Nullable final Long deviceId = CONTEXT_AUTHENTICATED_DEVICE_IDENTIFIER_KEY.get();
@Nullable final AuthenticatedDevice authenticatedDevice = CONTEXT_AUTHENTICATED_DEVICE.get();
if (accountIdentifier != null && deviceId != null) {
return new AuthenticatedDevice(accountIdentifier, deviceId);
if (authenticatedDevice != null) {
return authenticatedDevice;
}
throw Status.UNAUTHENTICATED.asRuntimeException();
}
/**
* Returns the account/device authenticated in the current gRPC context or throws an "unauthenticated" exception if
* no authenticated account/device is available or "permission denied" if the authenticated device is not the primary
* device for the account.
*
* @return the account/device identifier authenticated in the current gRPC context
*
* @throws io.grpc.StatusRuntimeException with a status of {@code UNAUTHENTICATED} if no authenticated account/device
* could be retrieved from the current gRPC context or a status of {@code PERMISSION_DENIED} if the authenticated
* device is not the primary device for the authenticated account
*/
public static AuthenticatedDevice requireAuthenticatedPrimaryDevice() {
final AuthenticatedDevice authenticatedDevice = requireAuthenticatedDevice();
if (authenticatedDevice.deviceId() != Device.PRIMARY_ID) {
throw Status.PERMISSION_DENIED.asRuntimeException();
}
return authenticatedDevice;
}
}

View File

@@ -1,94 +0,0 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth.grpc;
import com.google.common.annotations.VisibleForTesting;
import io.dropwizard.auth.basic.BasicCredentials;
import io.grpc.Context;
import io.grpc.Contexts;
import io.grpc.Metadata;
import io.grpc.ServerCall;
import io.grpc.ServerCallHandler;
import io.grpc.ServerInterceptor;
import io.grpc.Status;
import java.util.Optional;
import org.apache.commons.lang3.StringUtils;
import org.whispersystems.textsecuregcm.auth.AuthenticatedAccount;
import org.whispersystems.textsecuregcm.auth.BaseAccountAuthenticator;
/**
* A basic credential authentication interceptor enforces the presence of a valid username and password on every call.
* Callers supply credentials by providing a username (UUID and optional device ID) and password pair in the
* {@code x-signal-basic-auth-credentials} call header.
* <p/>
* Downstream services can retrieve the identity of the authenticated caller using methods in
* {@link AuthenticationUtil}.
* <p/>
* Note that this authentication, while fully functional, is intended only for development and testing purposes and is
* intended to be replaced with a more robust and efficient strategy before widespread client adoption.
*
* @see AuthenticationUtil
* @see BaseAccountAuthenticator
*/
public class BasicCredentialAuthenticationInterceptor implements ServerInterceptor {
private final BaseAccountAuthenticator baseAccountAuthenticator;
@VisibleForTesting
static final Metadata.Key<String> BASIC_CREDENTIALS =
Metadata.Key.of("x-signal-basic-auth-credentials", Metadata.ASCII_STRING_MARSHALLER);
private static final Metadata EMPTY_TRAILERS = new Metadata();
public BasicCredentialAuthenticationInterceptor(final BaseAccountAuthenticator baseAccountAuthenticator) {
this.baseAccountAuthenticator = baseAccountAuthenticator;
}
@Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(final ServerCall<ReqT, RespT> call,
final Metadata headers,
final ServerCallHandler<ReqT, RespT> next) {
final String credentialString = headers.get(BASIC_CREDENTIALS);
if (StringUtils.isNotBlank(credentialString)) {
try {
final BasicCredentials credentials = extractBasicCredentials(credentialString);
final Optional<AuthenticatedAccount> maybeAuthenticatedAccount =
baseAccountAuthenticator.authenticate(credentials, false);
if (maybeAuthenticatedAccount.isPresent()) {
final AuthenticatedAccount authenticatedAccount = maybeAuthenticatedAccount.get();
final Context context = Context.current()
.withValue(AuthenticationUtil.CONTEXT_AUTHENTICATED_ACCOUNT_IDENTIFIER_KEY, authenticatedAccount.getAccount().getUuid())
.withValue(AuthenticationUtil.CONTEXT_AUTHENTICATED_DEVICE_IDENTIFIER_KEY, authenticatedAccount.getAuthenticatedDevice().getId());
return Contexts.interceptCall(context, call, headers, next);
} else {
call.close(Status.UNAUTHENTICATED.withDescription("Credentials not accepted"), EMPTY_TRAILERS);
}
} catch (final IllegalArgumentException e) {
call.close(Status.UNAUTHENTICATED.withDescription("Could not parse credentials"), EMPTY_TRAILERS);
}
} else {
call.close(Status.UNAUTHENTICATED.withDescription("No credentials provided"), EMPTY_TRAILERS);
}
return new ServerCall.Listener<>() {};
}
@VisibleForTesting
static BasicCredentials extractBasicCredentials(final String credentials) {
if (credentials.indexOf(':') < 0) {
throw new IllegalArgumentException("Credentials do not include a username and password part");
}
final String[] pieces = credentials.split(":", 2);
return new BasicCredentials(pieces[0], pieces[1]);
}
}

View File

@@ -0,0 +1,30 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth.grpc;
import io.grpc.Metadata;
import io.grpc.ServerCall;
import io.grpc.ServerCallHandler;
import io.grpc.ServerInterceptor;
import io.grpc.Status;
/**
* A "prohibit authentication" interceptor ensures that requests to endpoints that should be invoked anonymously do not
* contain an authorization header in the request metdata. Calls with an associated authenticated device are closed with
* an {@code UNAUTHENTICATED} status.
*/
public class ProhibitAuthenticationInterceptor implements ServerInterceptor {
@Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(final ServerCall<ReqT, RespT> call,
final Metadata headers, final ServerCallHandler<ReqT, RespT> next) {
final String authHeaderString = headers.get(Metadata.Key.of(RequireAuthenticationInterceptor.AUTHORIZATION_HEADER, Metadata.ASCII_STRING_MARSHALLER));
if (authHeaderString != null) {
call.close(Status.UNAUTHENTICATED.withDescription("authorization header forbidden"), new Metadata());
return new ServerCall.Listener<>() {};
}
return next.startCall(call, headers);
}
}

View File

@@ -0,0 +1,68 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.auth.grpc;
import io.dropwizard.auth.basic.BasicCredentials;
import io.grpc.Context;
import io.grpc.Contexts;
import io.grpc.Metadata;
import io.grpc.ServerCall;
import io.grpc.ServerCallHandler;
import io.grpc.ServerInterceptor;
import io.grpc.Status;
import java.util.Optional;
import org.whispersystems.textsecuregcm.auth.AccountAuthenticator;
import org.whispersystems.textsecuregcm.grpc.ServerInterceptorUtil;
import org.whispersystems.textsecuregcm.util.HeaderUtils;
/**
* A "require authentication" interceptor authenticates requests and attaches the {@link AuthenticatedDevice} to the
* current gRPC context. Calls without authentication or with invalid credentials are closed with an
* {@code UNAUTHENTICATED} status. If a call's authentication status cannot be determined (i.e. because the accounts
* database is unavailable), the interceptor will reject the call with a status of {@code UNAVAILABLE}.
*/
public class RequireAuthenticationInterceptor implements ServerInterceptor {
static final String AUTHORIZATION_HEADER = "authorization";
private final AccountAuthenticator authenticator;
public RequireAuthenticationInterceptor(final AccountAuthenticator authenticator) {
this.authenticator = authenticator;
}
@Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(final ServerCall<ReqT, RespT> call,
final Metadata headers, final ServerCallHandler<ReqT, RespT> next) {
final String authHeaderString = headers.get(
Metadata.Key.of(AUTHORIZATION_HEADER, Metadata.ASCII_STRING_MARSHALLER));
if (authHeaderString == null) {
return ServerInterceptorUtil.closeWithStatus(call,
Status.UNAUTHENTICATED.withDescription("missing authorization header"));
}
final Optional<BasicCredentials> basicCredentials = HeaderUtils.basicCredentialsFromAuthHeader(authHeaderString);
if (basicCredentials.isEmpty()) {
return ServerInterceptorUtil.closeWithStatus(call,
Status.UNAUTHENTICATED.withDescription("malformed authorization header"));
}
final Optional<org.whispersystems.textsecuregcm.auth.AuthenticatedDevice> authenticated =
authenticator.authenticate(basicCredentials.get());
if (authenticated.isEmpty()) {
return ServerInterceptorUtil.closeWithStatus(call,
Status.UNAUTHENTICATED.withDescription("invalid credentials"));
}
final AuthenticatedDevice authenticatedDevice = new AuthenticatedDevice(
authenticated.get().accountIdentifier(),
authenticated.get().deviceId());
return Contexts.interceptCall(Context.current()
.withValue(AuthenticationUtil.CONTEXT_AUTHENTICATED_DEVICE, authenticatedDevice),
call, headers, next);
}
}

View File

@@ -0,0 +1,369 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import io.grpc.Status;
import java.security.MessageDigest;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.stream.StreamSupport;
import javax.annotation.Nullable;
import org.signal.libsignal.zkgroup.GenericServerSecretParams;
import org.signal.libsignal.zkgroup.InvalidInputException;
import org.signal.libsignal.zkgroup.VerificationFailedException;
import org.signal.libsignal.zkgroup.backups.BackupAuthCredentialRequest;
import org.signal.libsignal.zkgroup.backups.BackupAuthCredentialResponse;
import org.signal.libsignal.zkgroup.backups.BackupCredentialType;
import org.signal.libsignal.zkgroup.backups.BackupLevel;
import org.signal.libsignal.zkgroup.receipts.ReceiptCredentialPresentation;
import org.signal.libsignal.zkgroup.receipts.ReceiptSerial;
import org.signal.libsignal.zkgroup.receipts.ServerZkReceiptOperations;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.auth.RedemptionRange;
import org.whispersystems.textsecuregcm.controllers.RateLimitExceededException;
import org.whispersystems.textsecuregcm.experiment.ExperimentEnrollmentManager;
import org.whispersystems.textsecuregcm.limits.RateLimiter;
import org.whispersystems.textsecuregcm.limits.RateLimiters;
import org.whispersystems.textsecuregcm.storage.Account;
import org.whispersystems.textsecuregcm.storage.AccountsManager;
import org.whispersystems.textsecuregcm.storage.Device;
import org.whispersystems.textsecuregcm.storage.RedeemedReceiptsManager;
import org.whispersystems.textsecuregcm.util.Util;
/**
* Issues ZK backup auth credentials for authenticated accounts
* <p>
* Authenticated callers can create ZK credentials that contain a blinded backup-id, so that they can later use that
* backup id without the verifier learning that the id is associated with this account.
* <p>
* First use {@link #commitBackupId} to provide a blinded backup-id. This is stored in durable storage. Then the caller
* can use {@link #getBackupAuthCredentials} to retrieve credentials that can subsequently be used to make anonymously
* authenticated requests against their backup-id.
*/
public class BackupAuthManager {
private static final Logger logger = LoggerFactory.getLogger(BackupAuthManager.class);
final static Duration MAX_REDEMPTION_DURATION = Duration.ofDays(7);
final static String BACKUP_MEDIA_EXPERIMENT_NAME = "backupMedia";
private final ExperimentEnrollmentManager experimentEnrollmentManager;
private final GenericServerSecretParams serverSecretParams;
private final ServerZkReceiptOperations serverZkReceiptOperations;
private final RedeemedReceiptsManager redeemedReceiptsManager;
private final Clock clock;
private final RateLimiters rateLimiters;
private final AccountsManager accountsManager;
public BackupAuthManager(
final ExperimentEnrollmentManager experimentEnrollmentManager,
final RateLimiters rateLimiters,
final AccountsManager accountsManager,
final ServerZkReceiptOperations serverZkReceiptOperations,
final RedeemedReceiptsManager redeemedReceiptsManager,
final GenericServerSecretParams serverSecretParams,
final Clock clock) {
this.experimentEnrollmentManager = experimentEnrollmentManager;
this.rateLimiters = rateLimiters;
this.accountsManager = accountsManager;
this.serverZkReceiptOperations = serverZkReceiptOperations;
this.redeemedReceiptsManager = redeemedReceiptsManager;
this.serverSecretParams = serverSecretParams;
this.clock = clock;
}
/**
* Store credential requests containing blinded backup-ids for future use.
*
* @param account The account using the backup-id
* @param device The device setting the account backup-id
* @param messagesBackupCredentialRequest A request containing the blinded backup-id the client will use to upload
* message backups
* @param mediaBackupCredentialRequest A request containing the blinded backup-id the client will use to upload
* media backups
* @return A future that completes when the credentialRequest has been stored
* @throws RateLimitExceededException If too many backup-ids have been committed
*/
public CompletableFuture<Void> commitBackupId(
final Account account,
final Device device,
final Optional<BackupAuthCredentialRequest> messagesBackupCredentialRequest,
final Optional<BackupAuthCredentialRequest> mediaBackupCredentialRequest) {
if (!device.isPrimary()) {
throw Status.PERMISSION_DENIED.withDescription("Only primary device can set backup-id").asRuntimeException();
}
if (messagesBackupCredentialRequest.isEmpty() && mediaBackupCredentialRequest.isEmpty()) {
throw Status.INVALID_ARGUMENT
.withDescription("Must set at least one of message/media credential requests")
.asRuntimeException();
}
final byte[] storedMessageCredentialRequest = account.getBackupCredentialRequest(BackupCredentialType.MESSAGES)
.orElse(null);
final byte[] storedMediaCredentialRequest = account.getBackupCredentialRequest(BackupCredentialType.MEDIA)
.orElse(null);
// If the provided credential request is null, we want to set to the existing request
final byte[] targetMessageCredentialRequest = messagesBackupCredentialRequest
.map(BackupAuthCredentialRequest::serialize)
.orElse(storedMessageCredentialRequest);
final byte[] targetMediaCredentialRequest = mediaBackupCredentialRequest
.map(BackupAuthCredentialRequest::serialize)
.orElse(storedMediaCredentialRequest);
final boolean requiresMessageRotation =
!MessageDigest.isEqual(targetMessageCredentialRequest, storedMessageCredentialRequest);
final boolean requiresMediaRotation =
!MessageDigest.isEqual(targetMediaCredentialRequest, storedMediaCredentialRequest);
if (!requiresMessageRotation && !requiresMediaRotation) {
// No need to update or enforce rate limits, this is the credential that the user has already
// committed to.
return CompletableFuture.completedFuture(null);
}
CompletableFuture<Void> rateLimitFuture = CompletableFuture.completedFuture(null);
if (requiresMessageRotation) {
rateLimitFuture = rateLimitFuture.thenCombine(
rateLimiters.forDescriptor(RateLimiters.For.SET_BACKUP_ID).validateAsync(account.getUuid()),
(_, _) -> null);
}
if (requiresMediaRotation && hasActiveVoucher(account)) {
rateLimitFuture = rateLimitFuture.thenCombine(
rateLimiters.forDescriptor(RateLimiters.For.SET_PAID_MEDIA_BACKUP_ID).validateAsync(account.getUuid()),
(_, _) -> null);
}
return rateLimitFuture.thenCompose(ignored -> this.accountsManager
.updateAsync(account, a ->
a.setBackupCredentialRequests(targetMessageCredentialRequest, targetMediaCredentialRequest))
.thenRun(Util.NOOP))
.toCompletableFuture();
}
public record BackupIdRotationLimit(boolean hasPermitsRemaining, Duration nextPermitAvailable) {}
public CompletionStage<BackupIdRotationLimit> checkBackupIdRotationLimit(final Account account) {
final RateLimiter messagesLimiter = rateLimiters.forDescriptor(RateLimiters.For.SET_BACKUP_ID);
final RateLimiter mediaLimiter = rateLimiters.forDescriptor(RateLimiters.For.SET_PAID_MEDIA_BACKUP_ID);
final boolean isPaid = hasActiveVoucher(account);
final CompletionStage<Boolean> hasSetMessagesPermits =
messagesLimiter.hasAvailablePermitsAsync(account.getUuid(), 1);
final CompletionStage<Boolean> hasSetMediaPermits = isPaid
? mediaLimiter.hasAvailablePermitsAsync(account.getUuid(), 1)
: CompletableFuture.completedFuture(true);
return hasSetMessagesPermits.thenCombine(hasSetMediaPermits, (hasMessage, hasMedia) -> {
if (hasMedia && hasMessage) {
return new BackupIdRotationLimit(true, Duration.ZERO);
} else {
final Duration timeToNextPermit = Collections.max(Arrays.asList(
messagesLimiter.config().permitRegenerationDuration(),
isPaid ? mediaLimiter.config().permitRegenerationDuration() : Duration.ZERO));
return new BackupIdRotationLimit(false, timeToNextPermit);
}
});
}
public record Credential(BackupAuthCredentialResponse credential, Instant redemptionTime) {}
/**
* Create a credential for every day between redemptionStart and redemptionEnd
* <p>
* This uses a {@link BackupAuthCredentialRequest} previous stored via {@link this#commitBackupId} to generate the
* credentials.
* <p>
* If the account has a BackupVoucher allowing access to paid backups, credentials with a redemptionTime before the
* voucher's expiration will include paid backup access. If the BackupVoucher exists but is already expired, this
* method will also remove the expired voucher from the account.
*
* @param account The account to create the credentials for
* @param credentialType The type of backup credentials to create
* @param redemptionRange The time range to return credentials for
* @return Credentials and the day on which they may be redeemed
*/
public CompletableFuture<List<Credential>> getBackupAuthCredentials(
final Account account,
final BackupCredentialType credentialType,
final RedemptionRange redemptionRange) {
// If the account has an expired payment, clear it before continuing
if (hasExpiredVoucher(account)) {
return accountsManager.updateAsync(account, a -> {
// Re-check in case we raced with an update
if (hasExpiredVoucher(a)) {
a.setBackupVoucher(null);
}
}).thenCompose(updated -> getBackupAuthCredentials(updated, credentialType, redemptionRange));
}
// fetch the blinded backup-id the account should have previously committed to
final byte[] committedBytes = account.getBackupCredentialRequest(credentialType)
.orElseThrow(() -> Status.NOT_FOUND.withDescription("No blinded backup-id has been added to the account").asRuntimeException());
try {
final BackupLevel defaultBackupLevel = configuredBackupLevel(account);
// create a credential for every day in the requested period
final BackupAuthCredentialRequest credentialReq = new BackupAuthCredentialRequest(committedBytes);
return CompletableFuture.completedFuture(StreamSupport.stream(redemptionRange.spliterator(), false)
.map(redemptionTime -> {
// Check if the account has a voucher that's good for a certain receiptLevel at redemption time, otherwise
// use the default receipt level
final BackupLevel backupLevel = storedBackupLevel(account, redemptionTime).orElse(defaultBackupLevel);
return new Credential(
credentialReq.issueCredential(redemptionTime, backupLevel, credentialType, serverSecretParams),
redemptionTime);
})
.toList());
} catch (InvalidInputException e) {
throw Status.INTERNAL
.withDescription("Could not deserialize stored request credential")
.withCause(e)
.asRuntimeException();
}
}
/**
* Redeem a receipt to enable paid backups on the account.
*
* @param account The account to enable backups on
* @param receiptCredentialPresentation A ZK receipt presentation proving payment
* @return A future that completes successfully when the account has been updated
*/
public CompletableFuture<Void> redeemReceipt(
final Account account,
final ReceiptCredentialPresentation receiptCredentialPresentation) {
try {
serverZkReceiptOperations.verifyReceiptCredentialPresentation(receiptCredentialPresentation);
} catch (VerificationFailedException e) {
throw Status.INVALID_ARGUMENT
.withDescription("receipt credential presentation verification failed")
.asRuntimeException();
}
final ReceiptSerial receiptSerial = receiptCredentialPresentation.getReceiptSerial();
final Instant receiptExpiration = Instant.ofEpochSecond(receiptCredentialPresentation.getReceiptExpirationTime());
if (clock.instant().isAfter(receiptExpiration)) {
throw Status.INVALID_ARGUMENT.withDescription("receipt is already expired").asRuntimeException();
}
final long receiptLevel = receiptCredentialPresentation.getReceiptLevel();
if (BackupLevelUtil.fromReceiptLevel(receiptLevel) != BackupLevel.PAID) {
throw Status.INVALID_ARGUMENT
.withDescription("server does not recognize the requested receipt level")
.asRuntimeException();
}
if (account.getBackupCredentialRequest(BackupCredentialType.MEDIA).isEmpty()) {
throw Status.ABORTED
.withDescription("account must have a backup-id commitment")
.asRuntimeException();
}
return redeemedReceiptsManager
.put(receiptSerial, receiptExpiration.getEpochSecond(), receiptLevel, account.getUuid())
.thenCompose(receiptAllowed -> {
if (!receiptAllowed) {
throw Status.INVALID_ARGUMENT
.withDescription("receipt serial is already redeemed")
.asRuntimeException();
}
return extendBackupVoucher(account, new Account.BackupVoucher(receiptLevel, receiptExpiration));
});
}
/**
* Extend the duration of the backup voucher on an account.
*
* @param account The account to update
* @param backupVoucher The backup voucher to apply to this account
* @return A future that completes once the account has been updated to have at least the level and expiration
* in the provided voucher.
*/
public CompletableFuture<Void> extendBackupVoucher(final Account account, final Account.BackupVoucher backupVoucher) {
return accountsManager.updateAsync(account, a -> {
// Receipt credential expirations must be day aligned. Make sure any manually set backupVoucher is also day
// aligned
final Account.BackupVoucher newPayment = new Account.BackupVoucher(
backupVoucher.receiptLevel(),
backupVoucher.expiration().truncatedTo(ChronoUnit.DAYS));
final Account.BackupVoucher existingPayment = a.getBackupVoucher();
a.setBackupVoucher(merge(existingPayment, newPayment));
}).thenRun(Util.NOOP);
}
private static Account.BackupVoucher merge(@Nullable final Account.BackupVoucher prev,
final Account.BackupVoucher next) {
if (prev == null) {
return next;
}
if (next.receiptLevel() != prev.receiptLevel()) {
return next;
}
// If the new payment has the same receipt level as the old, select the further out of the two expiration times
if (prev.expiration().isAfter(next.expiration())) {
// This should be fairly rare, either a client reused an old receipt or we reduced the validity period
logger.warn(
"Redeemed receipt with an expiration at {} when we've previously had a redemption with a later expiration {}",
next.expiration(), prev.expiration());
return prev;
}
return next;
}
private boolean hasActiveVoucher(final Account account) {
return account.getBackupVoucher() != null && clock.instant().isBefore(account.getBackupVoucher().expiration());
}
private boolean hasExpiredVoucher(final Account account) {
return account.getBackupVoucher() != null && !hasActiveVoucher(account);
}
/**
* Get the receipt level stored in the {@link Account.BackupVoucher} on the account if it's present and not expired.
*
* @param account The account to check
* @param redemptionTime The time to check against the expiration time
* @return The receipt level on the backup voucher, or empty if the account does not have one or it is expired
*/
private Optional<BackupLevel> storedBackupLevel(final Account account, final Instant redemptionTime) {
return Optional.ofNullable(account.getBackupVoucher())
.filter(backupVoucher -> !redemptionTime.isAfter(backupVoucher.expiration()))
.map(Account.BackupVoucher::receiptLevel)
.map(BackupLevelUtil::fromReceiptLevel);
}
/**
* Get the backup receipt level that should be used by default for this account determined via configuration.
*
* @param account the account to check
* @return The default receipt level that should be used for the account if the account does not have a
* BackupVoucher.
*/
private BackupLevel configuredBackupLevel(final Account account) {
return this.experimentEnrollmentManager.isEnrolled(account.getUuid(), BACKUP_MEDIA_EXPERIMENT_NAME)
? BackupLevel.PAID
: BackupLevel.FREE;
}
}

View File

@@ -0,0 +1,17 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import org.signal.libsignal.zkgroup.backups.BackupLevel;
public class BackupLevelUtil {
public static BackupLevel fromReceiptLevel(long receiptLevel) {
try {
return BackupLevel.fromValue(Math.toIntExact(receiptLevel));
} catch (ArithmeticException e) {
throw new IllegalArgumentException("Invalid receipt level: " + receiptLevel);
}
}
}

View File

@@ -0,0 +1,849 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import com.google.common.annotations.VisibleForTesting;
import io.dropwizard.util.DataSize;
import io.grpc.Status;
import io.micrometer.core.instrument.DistributionSummary;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.Tags;
import io.micrometer.core.instrument.Timer;
import java.security.SecureRandom;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.util.Base64;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.function.Function;
import org.apache.commons.lang3.StringUtils;
import org.signal.libsignal.protocol.ecc.ECKeyPair;
import org.signal.libsignal.protocol.ecc.ECPublicKey;
import org.signal.libsignal.zkgroup.GenericServerSecretParams;
import org.signal.libsignal.zkgroup.VerificationFailedException;
import org.signal.libsignal.zkgroup.backups.BackupAuthCredentialPresentation;
import org.signal.libsignal.zkgroup.backups.BackupCredentialType;
import org.signal.libsignal.zkgroup.backups.BackupLevel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.attachments.AttachmentGenerator;
import org.whispersystems.textsecuregcm.attachments.TusAttachmentGenerator;
import org.whispersystems.textsecuregcm.auth.AuthenticatedBackupUser;
import org.whispersystems.textsecuregcm.auth.ExternalServiceCredentials;
import org.whispersystems.textsecuregcm.auth.ExternalServiceCredentialsGenerator;
import org.whispersystems.textsecuregcm.configuration.dynamic.DynamicBackupConfiguration;
import org.whispersystems.textsecuregcm.configuration.dynamic.DynamicConfiguration;
import org.whispersystems.textsecuregcm.limits.RateLimiters;
import org.whispersystems.textsecuregcm.metrics.MetricsUtil;
import org.whispersystems.textsecuregcm.metrics.UserAgentTagUtil;
import org.whispersystems.textsecuregcm.securevaluerecovery.SecureValueRecoveryClient;
import org.whispersystems.textsecuregcm.storage.DynamicConfigurationManager;
import org.whispersystems.textsecuregcm.util.AsyncTimerUtil;
import org.whispersystems.textsecuregcm.util.ExceptionUtils;
import org.whispersystems.textsecuregcm.util.Pair;
import org.whispersystems.textsecuregcm.util.ua.UnrecognizedUserAgentException;
import org.whispersystems.textsecuregcm.util.ua.UserAgent;
import org.whispersystems.textsecuregcm.util.ua.UserAgentUtil;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.core.scheduler.Scheduler;
import javax.annotation.Nullable;
public class BackupManager {
static final String MESSAGE_BACKUP_NAME = "messageBackup";
public static final long MAX_MESSAGE_BACKUP_OBJECT_SIZE = DataSize.mebibytes(101).toBytes();
public static final long MAX_MEDIA_OBJECT_SIZE = DataSize.mebibytes(101).toBytes();
private static final String ZK_AUTHN_COUNTER_NAME = MetricsUtil.name(BackupManager.class, "authentication");
private static final String ZK_AUTHZ_FAILURE_COUNTER_NAME = MetricsUtil.name(BackupManager.class,
"authorizationFailure");
private static final String USAGE_RECALCULATION_COUNTER_NAME = MetricsUtil.name(BackupManager.class,
"usageRecalculation");
private static final String DELETE_COUNT_DISTRIBUTION_NAME = MetricsUtil.name(BackupManager.class,
"deleteCount");
private static final Timer SYNCHRONOUS_DELETE_TIMER =
Metrics.timer(MetricsUtil.name(BackupManager.class, "synchronousDelete"));
private static final String NUM_OBJECTS_SUMMARY_NAME = MetricsUtil.name(BackupManager.class, "numObjects");
private static final String BYTES_USED_SUMMARY_NAME = MetricsUtil.name(BackupManager.class, "bytesUsed");
private static final String SUCCESS_TAG_NAME = "success";
private static final String FAILURE_REASON_TAG_NAME = "reason";
private static final Logger log = LoggerFactory.getLogger(BackupManager.class);
private final BackupsDb backupsDb;
private final GenericServerSecretParams serverSecretParams;
private final RateLimiters rateLimiters;
private final TusAttachmentGenerator tusAttachmentGenerator;
private final Cdn3BackupCredentialGenerator cdn3BackupCredentialGenerator;
private final RemoteStorageManager remoteStorageManager;
private final SecureRandom secureRandom = new SecureRandom();
private final ExternalServiceCredentialsGenerator secureValueRecoveryBCredentialsGenerator;
private final SecureValueRecoveryClient secureValueRecoveryBClient;
private final Clock clock;
private final DynamicConfigurationManager<DynamicConfiguration> dynamicConfigurationManager;
public BackupManager(
final BackupsDb backupsDb,
final GenericServerSecretParams serverSecretParams,
final RateLimiters rateLimiters,
final TusAttachmentGenerator tusAttachmentGenerator,
final Cdn3BackupCredentialGenerator cdn3BackupCredentialGenerator,
final RemoteStorageManager remoteStorageManager,
final ExternalServiceCredentialsGenerator secureValueRecoveryBCredentialsGenerator,
final SecureValueRecoveryClient secureValueRecoveryBClient,
final Clock clock,
final DynamicConfigurationManager<DynamicConfiguration> dynamicConfigurationManager) {
this.backupsDb = backupsDb;
this.serverSecretParams = serverSecretParams;
this.rateLimiters = rateLimiters;
this.tusAttachmentGenerator = tusAttachmentGenerator;
this.cdn3BackupCredentialGenerator = cdn3BackupCredentialGenerator;
this.remoteStorageManager = remoteStorageManager;
this.secureValueRecoveryBClient = secureValueRecoveryBClient;
this.clock = clock;
this.secureValueRecoveryBCredentialsGenerator = secureValueRecoveryBCredentialsGenerator;
this.dynamicConfigurationManager = dynamicConfigurationManager;
}
/**
* Set the public key for the backup-id.
* <p>
* Once set, calls {@link BackupManager#authenticateBackupUser} can succeed if the presentation is signed with the
* private key corresponding to this public key.
*
* @param presentation a ZK credential presentation that encodes the backupId
* @param signature the signature of the presentation
* @param publicKey the public key of a key-pair that the presentation must be signed with
*/
public CompletableFuture<Void> setPublicKey(
final BackupAuthCredentialPresentation presentation,
final byte[] signature,
final ECPublicKey publicKey) {
// Note: this is a special case where we can't validate the presentation signature against the stored public key
// because we are currently setting it. We check against the provided public key, but we must also verify that
// there isn't an existing, different stored public key for the backup-id (verified with a condition expression)
final Pair<BackupCredentialType, BackupLevel> credentialTypeAndBackupLevel =
verifyPresentation(presentation).verifySignature(signature, publicKey);
return backupsDb.setPublicKey(presentation.getBackupId(), credentialTypeAndBackupLevel.second(), publicKey)
.exceptionally(ExceptionUtils.exceptionallyHandler(PublicKeyConflictException.class, ex -> {
Metrics.counter(ZK_AUTHN_COUNTER_NAME,
SUCCESS_TAG_NAME, String.valueOf(false),
FAILURE_REASON_TAG_NAME, "public_key_conflict")
.increment();
throw Status.UNAUTHENTICATED
.withDescription("public key does not match existing public key for the backup-id")
.asRuntimeException();
}));
}
/**
* Create a form that may be used to upload a backup file for the backupId encoded in the presentation.
* <p>
* If successful, this also updates the TTL of the backup.
*
* @param backupUser an already ZK authenticated backup user
* @return the upload form
*/
public CompletableFuture<BackupUploadDescriptor> createMessageBackupUploadDescriptor(
final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.FREE);
checkBackupCredentialType(backupUser, BackupCredentialType.MESSAGES);
// this could race with concurrent updates, but the only effect would be last-writer-wins on the timestamp
return backupsDb
.addMessageBackup(backupUser)
.thenApply(_ ->
cdn3BackupCredentialGenerator.generateUpload(cdnMessageBackupName(backupUser)));
}
public CompletableFuture<BackupUploadDescriptor> createTemporaryAttachmentUploadDescriptor(
final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.PAID);
checkBackupCredentialType(backupUser, BackupCredentialType.MEDIA);
return rateLimiters.forDescriptor(RateLimiters.For.BACKUP_ATTACHMENT)
.validateAsync(rateLimitKey(backupUser)).thenApply(ignored -> {
final byte[] bytes = new byte[15];
secureRandom.nextBytes(bytes);
final String attachmentKey = Base64.getUrlEncoder().encodeToString(bytes);
final AttachmentGenerator.Descriptor descriptor = tusAttachmentGenerator.generateAttachment(attachmentKey);
return new BackupUploadDescriptor(3, attachmentKey, descriptor.headers(), descriptor.signedUploadLocation());
}).toCompletableFuture();
}
/**
* Update the last update timestamps for the backupId in the presentation
*
* @param backupUser an already ZK authenticated backup user
*/
public CompletableFuture<Void> ttlRefresh(final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.FREE);
// update message backup TTL
return backupsDb.ttlRefresh(backupUser).thenAccept(storedBackupAttributes -> {
if (backupUser.credentialType() == BackupCredentialType.MEDIA) {
final long maxTotalMediaSize =
dynamicConfigurationManager.getConfiguration().getBackupConfiguration().maxTotalMediaSize();
// Report that the backup is out of quota if it cannot store a max size media object
final boolean quotaExhausted = storedBackupAttributes.bytesUsed() >=
(maxTotalMediaSize - BackupManager.MAX_MEDIA_OBJECT_SIZE);
final Tags tags = Tags.of(
UserAgentTagUtil.getPlatformTag(backupUser.userAgent()),
Tag.of("type", backupUser.credentialType().name()),
Tag.of("tier", backupUser.backupLevel().name()),
Tag.of("quotaExhausted", String.valueOf(quotaExhausted)));
DistributionSummary.builder(NUM_OBJECTS_SUMMARY_NAME)
.tags(tags)
.publishPercentileHistogram()
.register(Metrics.globalRegistry)
.record(storedBackupAttributes.numObjects());
DistributionSummary.builder(BYTES_USED_SUMMARY_NAME)
.tags(tags)
.publishPercentileHistogram()
.register(Metrics.globalRegistry)
.record(storedBackupAttributes.bytesUsed());
}
});
}
public record BackupInfo(int cdn, String backupSubdir, String mediaSubdir, String messageBackupKey,
Optional<Long> mediaUsedSpace) {}
/**
* Retrieve information about the existing backup
*
* @param backupUser an already ZK authenticated backup user
* @return Information about the existing backup
*/
public CompletableFuture<BackupInfo> backupInfo(final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.FREE);
return backupsDb.describeBackup(backupUser)
.thenApply(backupDescription -> new BackupInfo(
backupDescription.cdn(),
backupUser.backupDir(),
backupUser.mediaDir(),
MESSAGE_BACKUP_NAME,
backupDescription.mediaUsedSpace()));
}
/**
* Copy an encrypted object to the backup cdn, adding a layer of encryption
* <p>
* Implementation notes: <p> This method guarantees that any object that gets successfully copied to the backup cdn
* will also be deducted from the user's quota. </p>
* <p>
* However, the converse isn't true. It's possible we may charge the user for media they failed to copy. As a result,
* the quota may be over reported. It should be recalculated before taking quota enforcement actions.
*
* @return A Flux that emits the locations of the double-encrypted objects on the backup cdn, or includes an error
* detailing why the object could not be copied.
*/
public Flux<CopyResult> copyToBackup(final AuthenticatedBackupUser backupUser, List<CopyParameters> toCopy) {
checkBackupLevel(backupUser, BackupLevel.PAID);
checkBackupCredentialType(backupUser, BackupCredentialType.MEDIA);
final DynamicBackupConfiguration backupConfiguration =
dynamicConfigurationManager.getConfiguration().getBackupConfiguration();
return Mono.fromFuture(() -> allowedCopies(backupUser, toCopy))
.flatMapMany(quotaResult -> Flux.concat(
// Perform copies for requests that fit in our quota, first updating the usage. If the copy fails, our
// estimated quota usage may not be exact since we update usage first. We make a best-effort attempt
// to undo the usage update if we know that the copied failed for sure.
Flux.fromIterable(quotaResult.requestsToCopy())
// Update the usage in reasonable chunk sizes to bound how out of sync our claimed and actual usage gets
.buffer(backupConfiguration.usageCheckpointCount())
.concatMap(copyParameters -> {
final long quotaToConsume = copyParameters.stream()
.mapToLong(CopyParameters::destinationObjectSize)
.sum();
return Mono
.fromFuture(backupsDb.trackMedia(backupUser, copyParameters.size(), quotaToConsume))
.thenMany(Flux.fromIterable(copyParameters));
})
// Actually perform the copies now that we've updated the quota
.flatMapSequential(copyParams -> copyToBackup(backupUser, copyParams)
.flatMap(copyResult -> switch (copyResult.outcome()) {
case SUCCESS -> Mono.just(copyResult);
case SOURCE_WRONG_LENGTH, SOURCE_NOT_FOUND, OUT_OF_QUOTA -> Mono
.fromFuture(this.backupsDb.trackMedia(backupUser, -1, -copyParams.destinationObjectSize()))
.thenReturn(copyResult);
}),
backupConfiguration.copyConcurrency(), 1),
// There wasn't enough quota remaining to perform these copies
Flux.fromIterable(quotaResult.requestsToReject())
.map(arg -> new CopyResult(CopyResult.Outcome.OUT_OF_QUOTA, arg.destinationMediaId(), null))
));
}
private Mono<CopyResult> copyToBackup(final AuthenticatedBackupUser backupUser, final CopyParameters copyParameters) {
return Mono.fromCompletionStage(() -> remoteStorageManager.copy(
copyParameters.sourceCdn(), copyParameters.sourceKey(), copyParameters.sourceLength(),
copyParameters.encryptionParameters(),
cdnMediaPath(backupUser, copyParameters.destinationMediaId())))
// Successfully copied!
.thenReturn(new CopyResult(
CopyResult.Outcome.SUCCESS, copyParameters.destinationMediaId(), remoteStorageManager.cdnNumber()))
// Otherwise, squash per-item copy errors that don't fail the entire operation
.onErrorResume(
// If the error maps to an explicit result type
throwable ->
CopyResult.fromCopyError(throwable, copyParameters.destinationMediaId()).isPresent(),
// return that result type instead of propagating the error
throwable ->
Mono.just(CopyResult.fromCopyError(throwable, copyParameters.destinationMediaId()).orElseThrow()));
}
private record QuotaResult(List<CopyParameters> requestsToCopy, List<CopyParameters> requestsToReject) {}
/**
* Determine which copy requests can be performed with the user's remaining quota. This does not update the quota.
*
* @param backupUser The user quota to check against
* @param toCopy The proposed copy requests
* @return list of QuotaResult indicating which requests fit into the remaining quota and which requests should be
* rejected with {@link CopyResult.Outcome#OUT_OF_QUOTA}
*/
private CompletableFuture<QuotaResult> allowedCopies(
final AuthenticatedBackupUser backupUser,
final List<CopyParameters> toCopy) {
final long totalBytesAdded = toCopy.stream()
.mapToLong(copyParameters -> {
if (copyParameters.sourceLength() > MAX_MEDIA_OBJECT_SIZE || copyParameters.sourceLength() < 0) {
throw Status.INVALID_ARGUMENT
.withDescription("Invalid sourceObject size")
.asRuntimeException();
}
return copyParameters.destinationObjectSize();
})
.sum();
final DynamicBackupConfiguration backupConfiguration =
dynamicConfigurationManager.getConfiguration().getBackupConfiguration();
final Duration maxQuotaStaleness = backupConfiguration.maxQuotaStaleness();
final long maxTotalMediaSize = backupConfiguration.maxTotalMediaSize();
return backupsDb.getMediaUsage(backupUser)
.thenComposeAsync(info -> {
long remainingQuota = maxTotalMediaSize - info.usageInfo().bytesUsed();
final boolean canStore = remainingQuota >= totalBytesAdded;
if (canStore || info.lastRecalculationTime().isAfter(clock.instant().minus(maxQuotaStaleness))) {
return CompletableFuture.completedFuture(remainingQuota);
}
// The user is out of quota, and we have not recently recalculated the user's usage. Double check by doing a
// hard recalculation before actually forbidding the user from storing additional media.
return this.remoteStorageManager.calculateBytesUsed(cdnMediaDirectory(backupUser))
.thenCompose(usage -> backupsDb
.setMediaUsage(backupUser, usage)
.thenApply(ignored -> usage))
.whenComplete((newUsage, throwable) -> {
boolean usageChanged = throwable == null && !newUsage.equals(info.usageInfo());
Metrics.counter(USAGE_RECALCULATION_COUNTER_NAME, Tags.of(
UserAgentTagUtil.getPlatformTag(backupUser.userAgent()),
Tag.of("usageChanged", String.valueOf(usageChanged))))
.increment();
})
.thenApply(newUsage -> maxTotalMediaSize - newUsage.bytesUsed());
})
.thenApply(remainingQuota -> {
// Figure out how many of the requested objects fit in the remaining quota
final int index = indexWhereTotalExceeds(toCopy, CopyParameters::destinationObjectSize,
remainingQuota);
return new QuotaResult(toCopy.subList(0, index), toCopy.subList(index, toCopy.size()));
});
}
public record RecalculationResult(UsageInfo oldUsage, UsageInfo newUsage) {}
public CompletionStage<Optional<RecalculationResult>> recalculateQuota(final StoredBackupAttributes storedBackupAttributes) {
if (StringUtils.isBlank(storedBackupAttributes.backupDir()) || StringUtils.isBlank(storedBackupAttributes.mediaDir())) {
return CompletableFuture.completedFuture(Optional.empty());
}
final String cdnPath = cdnMediaDirectory(storedBackupAttributes.backupDir(), storedBackupAttributes.mediaDir());
return this.remoteStorageManager.calculateBytesUsed(cdnPath).thenCompose(usage ->
backupsDb.setMediaUsage(storedBackupAttributes, usage).thenApply(ignored ->
Optional.of(new RecalculationResult(
new UsageInfo(storedBackupAttributes.bytesUsed(), storedBackupAttributes.numObjects()),
usage))));
}
/**
* @return the largest index i such that sum(ts[0],...ts[i - 1]) <= max
*/
private static <T> int indexWhereTotalExceeds(List<T> ts, Function<T, Long> valueFunction, long max) {
long sum = 0;
for (int index = 0; index < ts.size(); index++) {
sum += valueFunction.apply(ts.get(index));
if (sum > max) {
return index;
}
}
return ts.size();
}
public record StorageDescriptor(int cdn, byte[] key) {}
public record StorageDescriptorWithLength(int cdn, byte[] key, long length) {}
/**
* Generate credentials that can be used to read from the backup CDN
*
* @param backupUser an already ZK authenticated backup user
* @param cdnNumber the cdn number to get backup credentials for
* @return A map of headers to include with CDN requests
*/
public Map<String, String> generateReadAuth(final AuthenticatedBackupUser backupUser, final int cdnNumber) {
checkBackupLevel(backupUser, BackupLevel.FREE);
if (cdnNumber != 3) {
throw Status.INVALID_ARGUMENT.withDescription("unknown cdn").asRuntimeException();
}
return cdn3BackupCredentialGenerator.readHeaders(backupUser.backupDir());
}
/**
* Generate credentials that can be used with SVRB
*
* @param backupUser an already ZK authenticated backup user
* @return the credential that may be used with SVRB
*/
public ExternalServiceCredentials generateSvrbAuth(final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.FREE);
// Clients may only use SVRB with their messages backup-id
checkBackupCredentialType(backupUser, BackupCredentialType.MESSAGES);
return secureValueRecoveryBCredentialsGenerator.generateFor(svrbIdentifier(backupUser));
}
private static String svrbIdentifier(final AuthenticatedBackupUser backupUser) {
return svrbIdentifier(BackupsDb.hashedBackupId(backupUser.backupId()));
}
private static String svrbIdentifier(final byte[] hashedBackupId) {
return HexFormat.of().formatHex(hashedBackupId);
}
/**
* List of media stored for a particular backup id
*
* @param media A page of media entries
* @param cursor If set, can be passed back to a subsequent list request to resume listing from the previous point
*/
public record ListMediaResult(List<StorageDescriptorWithLength> media, Optional<String> cursor) {}
/**
* List the media stored by the backupUser
*
* @param backupUser An already ZK authenticated backup user
* @param cursor A cursor returned by a previous call that can be used to resume listing
* @param limit The maximum number of list results to return
* @return A {@link ListMediaResult}
*/
public CompletionStage<ListMediaResult> list(
final AuthenticatedBackupUser backupUser,
final Optional<String> cursor,
final int limit) {
checkBackupLevel(backupUser, BackupLevel.FREE);
return remoteStorageManager.list(cdnMediaDirectory(backupUser), cursor, limit)
.thenApply(result ->
new ListMediaResult(
result
.objects()
.stream()
.map(entry -> new StorageDescriptorWithLength(
remoteStorageManager.cdnNumber(),
decodeMediaIdFromCdn(entry.key()),
entry.length()
))
.toList(),
result.cursor()
));
}
public CompletableFuture<Void> deleteEntireBackup(final AuthenticatedBackupUser backupUser) {
checkBackupLevel(backupUser, BackupLevel.FREE);
final int deletionConcurrency =
dynamicConfigurationManager.getConfiguration().getBackupConfiguration().deletionConcurrency();
// Clients only include SVRB data with their messages backup-id
final CompletableFuture<Void> svrbRemoval = switch(backupUser.credentialType()) {
case BackupCredentialType.MESSAGES -> secureValueRecoveryBClient.removeData(svrbIdentifier(backupUser));
case BackupCredentialType.MEDIA -> CompletableFuture.completedFuture(null);
};
return svrbRemoval.thenCompose(_ -> backupsDb
// Try to swap out the backupDir for the user
.scheduleBackupDeletion(backupUser)
// If there was already a pending swap, try to delete the cdn objects directly
.exceptionallyCompose(ExceptionUtils.exceptionallyHandler(BackupsDb.PendingDeletionException.class, e ->
AsyncTimerUtil.record(SYNCHRONOUS_DELETE_TIMER, () ->
deletePrefix(backupUser.backupDir(), deletionConcurrency)))));
}
public Flux<StorageDescriptor> deleteMedia(final AuthenticatedBackupUser backupUser,
final List<StorageDescriptor> storageDescriptors) {
checkBackupLevel(backupUser, BackupLevel.FREE);
checkBackupCredentialType(backupUser, BackupCredentialType.MEDIA);
// Check for a cdn we don't know how to process
if (storageDescriptors.stream().anyMatch(sd -> sd.cdn() != remoteStorageManager.cdnNumber())) {
throw Status.INVALID_ARGUMENT
.withDescription("unsupported media cdn provided")
.asRuntimeException();
}
final DynamicBackupConfiguration backupConfiguration =
dynamicConfigurationManager.getConfiguration().getBackupConfiguration();
return Flux.usingWhen(
// Gather usage updates into the UsageBatcher so we don't have to update our backup record on every delete
Mono.just(new UsageBatcher(backupConfiguration.usageCheckpointCount())),
// Deletes the objects, returning their former location. Tracks bytes removed so the quota can be updated on
// completion
batcher -> Flux.fromIterable(storageDescriptors)
// Delete the objects, allowing DELETION_CONCURRENCY operations out at a time
.flatMapSequential(
sd -> Mono.fromCompletionStage(remoteStorageManager.delete(cdnMediaPath(backupUser, sd.key()))),
backupConfiguration.deletionConcurrency())
.zipWithIterable(storageDescriptors)
// Track how much the remote storage manager indicated was deleted as part of the operation
.concatMap(deletedBytesAndStorageDescriptor -> {
final long deletedBytes = deletedBytesAndStorageDescriptor.getT1();
final StorageDescriptor sd = deletedBytesAndStorageDescriptor.getT2();
// If it has been a while, perform a checkpoint to make sure our usage doesn't drift too much
if (batcher.update(-deletedBytes)) {
final UsageBatcher.UsageUpdate usageUpdate = batcher.getAndReset();
return Mono
.fromFuture(backupsDb.trackMedia(backupUser, usageUpdate.countDelta, usageUpdate.bytesDelta))
.doOnError(throwable ->
log.warn("Failed to update delta {} after successful delete operation", usageUpdate, throwable))
.thenReturn(sd);
} else {
return Mono.just(sd);
}
}),
// On cleanup, update the quota using whatever remaining updates were accumulated in the batcher
batcher -> {
final UsageBatcher.UsageUpdate update = batcher.getAndReset();
return Mono
.fromFuture(backupsDb.trackMedia(backupUser, update.countDelta, update.bytesDelta))
.doOnError(throwable ->
log.warn("Failed to update delta {} after successful delete operation", update, throwable));
});
}
/**
* Track pending media usage updates. Not thread safe!
*/
private static class UsageBatcher {
private final int usageCheckpointCount;
private long runningCountDelta = 0;
private long runningBytesDelta = 0;
UsageBatcher(int usageCheckpointCount) {
this.usageCheckpointCount = usageCheckpointCount;
}
record UsageUpdate(long countDelta, long bytesDelta) {}
/**
* Stage a usage update. Returns true when it is time to make a checkpoint
*
* @param bytesDelta The amount of bytes that should be tracked as used (or if negative, freed). If the delta is
* non-zero, the count will also be updated.
* @return true if we should persist the usage
*/
boolean update(long bytesDelta) {
this.runningCountDelta += Long.signum(bytesDelta);
this.runningBytesDelta += bytesDelta;
return Math.abs(runningCountDelta) >= usageCheckpointCount;
}
/**
* Get the current usage delta, and set the delta to 0
* @return A {@link UsageUpdate} to apply
*/
UsageUpdate getAndReset() {
final UsageUpdate update = new UsageUpdate(runningCountDelta, runningBytesDelta);
runningCountDelta = 0;
runningBytesDelta = 0;
return update;
}
}
private static final ECPublicKey INVALID_PUBLIC_KEY = ECKeyPair.generate().getPublicKey();
/**
* Authenticate the ZK anonymous backup credential's presentation
* <p>
* This validates:
* <li> The presentation was for a credential issued by the server </li>
* <li> The credential is in its redemption window </li>
* <li> The backup-id matches a previously committed blinded backup-id and server issued receipt level </li>
* <li> The signature of the credential matches an existing publicKey associated with this backup-id </li>
*
* @param presentation A {@link BackupAuthCredentialPresentation}
* @param signature An XEd25519 signature of the presentation bytes
* @return On authentication success, the authenticated backup-id and backup-tier encoded in the presentation
*/
public CompletableFuture<AuthenticatedBackupUser> authenticateBackupUser(
final BackupAuthCredentialPresentation presentation,
final byte[] signature,
final String userAgentString) {
final PresentationSignatureVerifier signatureVerifier = verifyPresentation(presentation);
return backupsDb
.retrieveAuthenticationData(presentation.getBackupId())
.thenApply(optionalAuthenticationData -> {
final UserAgent userAgent = parseUserAgent(userAgentString);
final BackupsDb.AuthenticationData authenticationData = optionalAuthenticationData
.orElseGet(() -> {
Metrics.counter(ZK_AUTHN_COUNTER_NAME, Tags.of(
Tag.of(SUCCESS_TAG_NAME, String.valueOf(false)),
Tag.of(FAILURE_REASON_TAG_NAME, "missing_public_key"),
UserAgentTagUtil.getPlatformTag(userAgent)))
.increment();
// There was no stored public key, use a bunk public key so that validation will fail
return new BackupsDb.AuthenticationData(INVALID_PUBLIC_KEY, null, null);
});
final Pair<BackupCredentialType, BackupLevel> credentialTypeAndBackupLevel =
signatureVerifier.verifySignature(signature, authenticationData.publicKey());
return new AuthenticatedBackupUser(
presentation.getBackupId(),
credentialTypeAndBackupLevel.first(),
credentialTypeAndBackupLevel.second(),
authenticationData.backupDir(),
authenticationData.mediaDir(),
userAgent);
})
.thenApply(result -> {
Metrics.counter(ZK_AUTHN_COUNTER_NAME, SUCCESS_TAG_NAME, String.valueOf(true)).increment();
return result;
});
}
/**
* List all backups stored in the backups table
*
* @param segments Number of segments to read in parallel from the underlying backup database
* @return Flux of {@link StoredBackupAttributes} for each backup record in the backups table
*/
public Flux<StoredBackupAttributes> listBackupAttributes(final int segments) {
return this.backupsDb.listBackupAttributes(segments);
}
/**
* List all backups whose media or messages refresh timestamp are older than the provided purgeTime
*
* @param segments Number of segments to read in parallel from the underlying backup database
* @param scheduler Scheduler for running downstream operations
* @param purgeTime If a backup's last message refresh time is strictly before purgeTime, it will be marked as
* requiring full deletion. If only the last refresh time is strictly before purgeTime, it will be
* marked as requiring message deletion. Otherwise, it will not be included in the results.
* @return Flux of backups that require some deletion action
*/
public Flux<ExpiredBackup> getExpiredBackups(final int segments, final Scheduler scheduler, final Instant purgeTime) {
return this.backupsDb.getExpiredBackups(segments, scheduler, purgeTime);
}
/**
* Delete some or all of the objects associated with the backup, and update the backup database.
*
* @param expiredBackup The backup to expire. If the {@link ExpiredBackup} is a media expiration, only the media
* objects will be deleted, otherwise all backup objects will be deleted
* @return A stage that completes when the deletion operation is finished
*/
public CompletableFuture<Void> expireBackup(final ExpiredBackup expiredBackup) {
// Clients only include SVRB data with their messages backup-id
final CompletableFuture<Void> svrbRemoval = switch(expiredBackup.expirationType()) {
case ALL -> secureValueRecoveryBClient.removeData(svrbIdentifier(expiredBackup.hashedBackupId()));
case MEDIA, GARBAGE_COLLECTION -> CompletableFuture.completedFuture(null);
};
return svrbRemoval.thenCompose(_ -> backupsDb.startExpiration(expiredBackup)
// the deletion operation is effectively single threaded -- it's expected that the caller can increase
// concurrency by deleting more backups at once, rather than increasing concurrency deleting an individual
// backup
.thenCompose(ignored -> deletePrefix(expiredBackup.prefixToDelete(), 1))
.thenCompose(ignored -> backupsDb.finishExpiration(expiredBackup)));
}
/**
* List and delete all files associated with a prefix
*
* @param prefixToDelete The prefix to expire.
* @return A stage that completes when all objects with the given prefix have been deleted
*/
private CompletableFuture<Void> deletePrefix(final String prefixToDelete, int concurrentDeletes) {
if (prefixToDelete.length() != BackupsDb.BACKUP_DIRECTORY_PATH_LENGTH
&& prefixToDelete.length() != BackupsDb.MEDIA_DIRECTORY_PATH_LENGTH) {
throw new IllegalArgumentException("Unexpected prefix deletion for " + prefixToDelete);
}
final String prefix = prefixToDelete + "/";
return Mono
.fromCompletionStage(this.remoteStorageManager.list(prefix, Optional.empty(), 1000))
.expand(listResult -> {
if (listResult.cursor().isEmpty()) {
return Mono.empty();
}
return Mono.fromCompletionStage(() -> this.remoteStorageManager.list(prefix, listResult.cursor(), 1000));
})
.flatMap(listResult -> Flux.fromIterable(listResult.objects()))
.flatMap(
result -> Mono.fromCompletionStage(() -> remoteStorageManager.delete(prefix + result.key())),
concurrentDeletes)
.count()
.doOnSuccess(itemsRemoved -> DistributionSummary.builder(DELETE_COUNT_DISTRIBUTION_NAME)
.publishPercentileHistogram(true)
.register(Metrics.globalRegistry)
.record(itemsRemoved))
.then()
.toFuture();
}
interface PresentationSignatureVerifier {
Pair<BackupCredentialType, BackupLevel> verifySignature(byte[] signature, ECPublicKey publicKey);
}
/**
* Verify the presentation was issued by us, which should be done before checking the stored public key
*
* @param presentation A ZK credential presentation that encodes the backupId and the receipt level of the requester
* @return A function that can be used to verify a signature provided with the presentation
*/
private PresentationSignatureVerifier verifyPresentation(final BackupAuthCredentialPresentation presentation) {
try {
presentation.verify(clock.instant(), serverSecretParams);
} catch (VerificationFailedException e) {
Metrics.counter(ZK_AUTHN_COUNTER_NAME,
SUCCESS_TAG_NAME, String.valueOf(false),
FAILURE_REASON_TAG_NAME, "presentation_verification")
.increment();
throw Status.UNAUTHENTICATED
.withDescription("backup auth credential presentation verification failed")
.withCause(e)
.asRuntimeException();
}
return (signature, publicKey) -> {
if (!publicKey.verifySignature(presentation.serialize(), signature)) {
Metrics.counter(ZK_AUTHN_COUNTER_NAME,
SUCCESS_TAG_NAME, String.valueOf(false),
FAILURE_REASON_TAG_NAME, "signature_validation")
.increment();
throw Status.UNAUTHENTICATED
.withDescription("backup auth credential presentation signature verification failed")
.asRuntimeException();
}
return new Pair<>(presentation.getType(), presentation.getBackupLevel());
};
}
/**
* Check that the authenticated backup user is authorized to use the provided backupLevel
*
* @param backupUser The backup user to check
* @param backupLevel The authorization level to verify the backupUser has access to
* @throws {@link Status#PERMISSION_DENIED} error if the backup user is not authorized to access {@code backupLevel}
*/
@VisibleForTesting
static void checkBackupLevel(final AuthenticatedBackupUser backupUser, final BackupLevel backupLevel) {
if (backupUser.backupLevel().compareTo(backupLevel) < 0) {
Metrics.counter(ZK_AUTHZ_FAILURE_COUNTER_NAME, Tags.of(
UserAgentTagUtil.getPlatformTag(backupUser.userAgent()),
Tag.of(FAILURE_REASON_TAG_NAME, "level")))
.increment();
throw Status.PERMISSION_DENIED
.withDescription("credential does not support the requested operation")
.asRuntimeException();
}
}
/**
* Check that the authenticated backup user is authenticated with the given credential type
*
* @param backupUser The backup user to check
* @param credentialType The credential type to require
* @throws {@link Status#UNAUTHENTICATED} error if the backup user is not authenticated with the given
* {@code credentialType}
*/
@VisibleForTesting
static void checkBackupCredentialType(final AuthenticatedBackupUser backupUser, final BackupCredentialType credentialType) {
if (backupUser.credentialType() != credentialType) {
Metrics.counter(ZK_AUTHZ_FAILURE_COUNTER_NAME,
FAILURE_REASON_TAG_NAME, "credential_type")
.increment();
throw Status.UNAUTHENTICATED
.withDescription("wrong credential type for the requested operation")
.asRuntimeException();
}
}
@VisibleForTesting
static String encodeMediaIdForCdn(final byte[] bytes) {
return Base64.getUrlEncoder().encodeToString(bytes);
}
private static byte[] decodeMediaIdFromCdn(final String base64) {
return Base64.getUrlDecoder().decode(base64);
}
private static String cdnMessageBackupName(final AuthenticatedBackupUser backupUser) {
return "%s/%s".formatted(backupUser.backupDir(), MESSAGE_BACKUP_NAME);
}
private static String cdnMediaDirectory(final String backupDir, final String mediaDir) {
return "%s/%s/".formatted(backupDir, mediaDir);
}
private static String cdnMediaDirectory(final AuthenticatedBackupUser backupUser) {
return cdnMediaDirectory(backupUser.backupDir(), backupUser.mediaDir());
}
private static String cdnMediaPath(final AuthenticatedBackupUser backupUser, final byte[] mediaId) {
return "%s%s".formatted(cdnMediaDirectory(backupUser), encodeMediaIdForCdn(mediaId));
}
static String rateLimitKey(final AuthenticatedBackupUser backupUser) {
return Base64.getEncoder().encodeToString(BackupsDb.hashedBackupId(backupUser.backupId()));
}
private static @Nullable UserAgent parseUserAgent(final String userAgentString) {
try {
return UserAgentUtil.parseUserAgentString(userAgentString);
} catch (UnrecognizedUserAgentException e) {
return null;
}
}
}

View File

@@ -0,0 +1,14 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import java.util.Map;
public record BackupUploadDescriptor(
int cdn,
String key,
Map<String, String> headers,
String signedUploadLocation) {}

View File

@@ -0,0 +1,833 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import io.grpc.Status;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.time.Clock;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.function.Predicate;
import io.micrometer.core.instrument.DistributionSummary;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.Tags;
import org.signal.libsignal.protocol.InvalidKeyException;
import org.signal.libsignal.protocol.ecc.ECPublicKey;
import org.signal.libsignal.zkgroup.backups.BackupLevel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.auth.AuthenticatedBackupUser;
import org.whispersystems.textsecuregcm.metrics.MetricsUtil;
import org.whispersystems.textsecuregcm.metrics.UserAgentTagUtil;
import org.whispersystems.textsecuregcm.util.AttributeValues;
import org.whispersystems.textsecuregcm.util.ExceptionUtils;
import org.whispersystems.textsecuregcm.util.Util;
import reactor.core.publisher.Flux;
import reactor.core.scheduler.Scheduler;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException;
import software.amazon.awssdk.services.dynamodb.model.DeleteItemRequest;
import software.amazon.awssdk.services.dynamodb.model.GetItemRequest;
import software.amazon.awssdk.services.dynamodb.model.ReturnValue;
import software.amazon.awssdk.services.dynamodb.model.ScanRequest;
import software.amazon.awssdk.services.dynamodb.model.Update;
import software.amazon.awssdk.services.dynamodb.model.UpdateItemRequest;
/**
* Tracks backup metadata in a persistent store.
* <p>
* It's assumed that the caller has already validated that the backupUser being operated on has valid credentials and
* possesses the appropriate {@link BackupLevel} to perform the current operation.
* <p>
* Backup records track two timestamps indicating the last time that a user interacted with their backup. One for the
* last refresh that contained a credential including media level, and the other for any access. After a period of
* inactivity stale backups can be purged (either just the media, or the entire backup). Callers can discover what
* backups are stale and whether only the media or the entire backup is stale via {@link #getExpiredBackups}.
* <p>
* Because backup objects reside on a transactionally unrelated store, expiring anything from the backup requires a 2
* phase process. First the caller calls {@link #startExpiration} which will atomically update the user's backup
* directories and record the cdn directory that should be expired. Then the caller must delete the expired directory,
* calling {@link #finishExpiration} to clear the recorded expired prefix when complete. Since the user's backup
* directories have been swapped, the deleter does not have to account for a user coming back and starting to upload
* concurrently with the deletion.
* <p>
* If the directory deletion fails, a subsequent call to {@link #getExpiredBackups} will return the backup again
* indicating that the old expired prefix needs to be cleaned up before any other expiration action is taken. For
* example, if a media expiration fails and then in the next expiration pass the backup has become eligible for total
* deletion, the caller still must process the stale media expiration first before processing the full deletion.
*/
public class BackupsDb {
private static final int DIR_NAME_LENGTH = generateDirName(new SecureRandom()).length();
public static final int BACKUP_DIRECTORY_PATH_LENGTH = DIR_NAME_LENGTH;
public static final int MEDIA_DIRECTORY_PATH_LENGTH = BACKUP_DIRECTORY_PATH_LENGTH + "/".length() + DIR_NAME_LENGTH;
private static final Logger logger = LoggerFactory.getLogger(BackupsDb.class);
static final int BACKUP_CDN = 3;
private final DynamoDbAsyncClient dynamoClient;
private final String backupTableName;
private final Clock clock;
private final SecureRandom secureRandom;
// The backups table
// B: 16 bytes that identifies the backup
public static final String KEY_BACKUP_ID_HASH = "U";
// N: Time in seconds since epoch of the last backup refresh. This timestamp must be periodically updated to avoid
// garbage collection of archive objects.
public static final String ATTR_LAST_REFRESH = "R";
// N: Time in seconds since epoch of the last backup media refresh. This timestamp can only be updated if the client
// has BackupLevel.PAID, and must be periodically updated to avoid garbage collection of media objects.
public static final String ATTR_LAST_MEDIA_REFRESH = "MR";
// B: A 32 byte public key that should be used to sign the presentation used to authenticate requests against the
// backup-id
public static final String ATTR_PUBLIC_KEY = "P";
// N: Bytes consumed by this backup
public static final String ATTR_MEDIA_BYTES_USED = "MB";
// N: Number of media objects in the backup
public static final String ATTR_MEDIA_COUNT = "MC";
// N: The cdn number where the message backup is stored
public static final String ATTR_CDN = "CDN";
// N: Time in seconds since epoch of last backup media usage recalculation. This timestamp is updated whenever we
// recalculate the up-to-date bytes used by querying the cdn(s) directly.
public static final String ATTR_MEDIA_USAGE_LAST_RECALCULATION = "MBTS";
// S: The name of the user's backup directory on the CDN
public static final String ATTR_BACKUP_DIR = "BD";
// S: The name of the user's media directory within the backup directory on the CDN
public static final String ATTR_MEDIA_DIR = "MD";
// S: A prefix pending deletion
public static final String ATTR_EXPIRED_PREFIX = "EP";
public BackupsDb(
final DynamoDbAsyncClient dynamoClient,
final String backupTableName,
final Clock clock) {
this.dynamoClient = dynamoClient;
this.backupTableName = backupTableName;
this.clock = clock;
this.secureRandom = new SecureRandom();
}
/**
* Set the public key associated with a backupId.
*
* @param authenticatedBackupId The backup-id bytes that should be associated with the provided public key
* @param authenticatedBackupLevel The backup level
* @param publicKey The public key to associate with the backup id
* @return A stage that completes when the public key has been set. If the backup-id already has a set public key that
* does not match, the stage will be completed exceptionally with a {@link PublicKeyConflictException}
*/
CompletableFuture<Void> setPublicKey(
final byte[] authenticatedBackupId,
final BackupLevel authenticatedBackupLevel,
final ECPublicKey publicKey) {
final byte[] hashedBackupId = hashedBackupId(authenticatedBackupId);
return dynamoClient.updateItem(new UpdateBuilder(backupTableName, authenticatedBackupLevel, hashedBackupId)
.addSetExpression("#publicKey = :publicKey",
Map.entry("#publicKey", ATTR_PUBLIC_KEY),
Map.entry(":publicKey", AttributeValues.b(publicKey.serialize())))
// When the user sets a public key, we ensure that they have a backupDir/mediaDir assigned
.setDirectoryNamesIfMissing(secureRandom)
.setRefreshTimes(clock)
.withConditionExpression("attribute_not_exists(#publicKey) OR #publicKey = :publicKey")
.updateItemBuilder()
.build())
.exceptionally(ExceptionUtils.marshal(ConditionalCheckFailedException.class, e ->
// There was already a row for this backup-id and it contained a different publicKey
new PublicKeyConflictException()))
.thenRun(Util.NOOP);
}
/**
* Data stored to authenticate a backup user
*
* @param publicKey The public key for the backup entry. All credentials for this backup user must be signed * by this
* public key for the credential to be valid
* @param backupDir The current backupDir for the backup user. If authentication is successful, the user may be given
* credentials for this backupDir on the CDN
* @param mediaDir The current mediaDir for the backup user. If authentication is successful, the user may be given *
* credentials for the path backupDir/mediaDir on the CDN
*/
record AuthenticationData(ECPublicKey publicKey, String backupDir, String mediaDir) {}
CompletableFuture<Optional<AuthenticationData>> retrieveAuthenticationData(byte[] backupId) {
final byte[] hashedBackupId = hashedBackupId(backupId);
return dynamoClient.getItem(GetItemRequest.builder()
.tableName(backupTableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId)))
.consistentRead(true)
.projectionExpression("#publicKey,#backupDir,#mediaDir")
.expressionAttributeNames(Map.of(
"#publicKey", ATTR_PUBLIC_KEY,
"#backupDir", ATTR_BACKUP_DIR,
"#mediaDir", ATTR_MEDIA_DIR))
.build())
.thenApply(response -> extractStoredPublicKey(response.item())
.map(pubKey -> new AuthenticationData(
pubKey,
getDirName(response.item(), ATTR_BACKUP_DIR),
getDirName(response.item(), ATTR_MEDIA_DIR))));
}
private static String getDirName(final Map<String, AttributeValue> item, final String attr) {
return AttributeValues.get(item, attr).map(AttributeValue::s).orElseThrow(() -> {
logger.error("Backups with public keys should have directory names");
return Status.INTERNAL
.withDescription("Backups with public keys must have directory names")
.asRuntimeException();
});
}
private static Optional<ECPublicKey> extractStoredPublicKey(final Map<String, AttributeValue> item) {
return AttributeValues.get(item, ATTR_PUBLIC_KEY)
.map(AttributeValue::b)
.map(SdkBytes::asByteArray)
.map(BackupsDb::deserializeStoredPublicKey);
}
private static ECPublicKey deserializeStoredPublicKey(final byte[] publicKeyBytes) {
try {
return new ECPublicKey(publicKeyBytes);
} catch (InvalidKeyException e) {
logger.error("Invalid publicKey {}", HexFormat.of().formatHex(publicKeyBytes), e);
throw Status.INTERNAL
.withCause(e)
.withDescription("Could not deserialize stored public key")
.asRuntimeException();
}
}
/**
* Update the quota in the backup table
*
* @param backupUser The backup user
* @param mediaBytesDelta The length of the media after encryption. A negative length implies media being removed
* @param mediaCountDelta The number of media objects being added, or if negative, removed
* @return A stage that completes successfully once the table are updated.
*/
CompletableFuture<Void> trackMedia(final AuthenticatedBackupUser backupUser, final long mediaCountDelta,
final long mediaBytesDelta) {
return dynamoClient
.updateItem(
// Update the media quota and TTL
UpdateBuilder.forUser(backupTableName, backupUser)
.incrementMediaBytes(mediaBytesDelta)
.incrementMediaCount(mediaCountDelta)
.updateItemBuilder()
.build())
.thenRun(Util.NOOP);
}
/**
* Update the last update timestamps for the backupId in the presentation
*
* @param backupUser an already authorized backup user
*/
CompletableFuture<StoredBackupAttributes> ttlRefresh(final AuthenticatedBackupUser backupUser) {
final Instant today = clock.instant().truncatedTo(ChronoUnit.DAYS);
// update message backup TTL
return dynamoClient.updateItem(UpdateBuilder.forUser(backupTableName, backupUser)
.setRefreshTimes(today)
.updateItemBuilder()
.returnValues(ReturnValue.ALL_OLD)
.build())
.thenApply(updateItemResponse -> fromItem(updateItemResponse.attributes()));
}
/**
* Track that a backup will be stored for the user
*
* @param backupUser an already authorized backup user
* @return A future that completes with the attributes of the backup before the update
*/
CompletableFuture<StoredBackupAttributes> addMessageBackup(final AuthenticatedBackupUser backupUser) {
final Instant today = clock.instant().truncatedTo(ChronoUnit.DAYS);
// this could race with concurrent updates, but the only effect would be last-writer-wins on the timestamp
return dynamoClient.updateItem(
UpdateBuilder.forUser(backupTableName, backupUser)
.setRefreshTimes(today)
.setCdn(BACKUP_CDN)
.updateItemBuilder()
.returnValues(ReturnValue.ALL_OLD)
.build())
.thenApply(updateItemResponse -> fromItem(updateItemResponse.attributes()));
}
/**
* Indicates that we couldn't schedule a deletion because one was already scheduled. The caller may want to delete the
* objects directly.
*/
static class PendingDeletionException extends IOException {}
/**
* Attempt to mark a backup as expired and swap in a new empty backupDir for the user.
* <p>
* After successful completion, the backupDir for the backup-id will be swapped to a new empty directory on the cdn,
* and the row will be immediately marked eligible for expiration via {@link #getExpiredBackups}.
* <p>
* If there is already a pending deletion, this will not swap the backupDir. The expiration timestamps will be
* updated, but the existing backupDir will remain. The caller should handle this case and start the deletion
* immediately by catching {@link PendingDeletionException}.
*
* @param backupUser The backupUser whose data should be eventually deleted
* @return A future that completes successfully if the user's data is now inaccessible, or with a
* {@link PendingDeletionException} if the backupDir could not be changed.
*/
CompletableFuture<Void> scheduleBackupDeletion(final AuthenticatedBackupUser backupUser) {
final byte[] hashedBackupId = hashedBackupId(backupUser);
// Clear usage metadata, swap names of things we intend to delete, and record our intent to delete in attr_expired_prefix
return dynamoClient.updateItem(new UpdateBuilder(backupTableName, BackupLevel.PAID, hashedBackupId)
.clearMediaUsage(clock)
.expireDirectoryNames(secureRandom, ExpiredBackup.ExpirationType.ALL)
.setRefreshTimes(Instant.ofEpochSecond(0))
.addSetExpression("#expiredPrefix = :expiredPrefix",
Map.entry("#expiredPrefix", ATTR_EXPIRED_PREFIX),
Map.entry(":expiredPrefix", AttributeValues.s(backupUser.backupDir())))
.withConditionExpression("attribute_not_exists(#expiredPrefix) OR #expiredPrefix = :expiredPrefix")
.updateItemBuilder()
.build())
.exceptionallyCompose(ExceptionUtils.exceptionallyHandler(ConditionalCheckFailedException.class, e ->
// We already have a pending deletion for this backup-id. This is most likely to occur when the caller
// is toggling backups on and off. In this case, it should be pretty cheap to directly delete the backup.
// Instead of changing the backupDir, just make sure the row has expired/ timestamps and tell the caller we
// couldn't schedule the deletion.
dynamoClient.updateItem(new UpdateBuilder(backupTableName, BackupLevel.PAID, hashedBackupId)
.setRefreshTimes(Instant.ofEpochSecond(0))
.updateItemBuilder()
.build())
.thenApply(ignore -> {
throw ExceptionUtils.wrap(new PendingDeletionException());
})))
.thenRun(Util.NOOP);
}
record BackupDescription(int cdn, Optional<Long> mediaUsedSpace) {}
/**
* Retrieve information about the backup
*
* @param backupUser an already authorized backup user
* @return A {@link BackupDescription} containing the cdn of the message backup and the total number of media space
* bytes used by the backup user.
*/
CompletableFuture<BackupDescription> describeBackup(final AuthenticatedBackupUser backupUser) {
return dynamoClient.getItem(GetItemRequest.builder()
.tableName(backupTableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId(backupUser))))
.projectionExpression("#cdn,#mediaBytesUsed")
.expressionAttributeNames(Map.of("#cdn", ATTR_CDN, "#mediaBytesUsed", ATTR_MEDIA_BYTES_USED))
.consistentRead(true)
.build())
.thenApply(response -> {
if (!response.hasItem()) {
throw Status.NOT_FOUND.withDescription("Backup ID not found").asRuntimeException();
}
// If the client hasn't already uploaded a backup, return the cdn we would return if they did create one
final int cdn = AttributeValues.getInt(response.item(), ATTR_CDN, BACKUP_CDN);
final Optional<Long> mediaUsed = AttributeValues.get(response.item(), ATTR_MEDIA_BYTES_USED)
.map(AttributeValue::n)
.map(Long::parseLong);
return new BackupDescription(cdn, mediaUsed);
});
}
public record TimestampedUsageInfo(UsageInfo usageInfo, Instant lastRecalculationTime) {}
CompletableFuture<TimestampedUsageInfo> getMediaUsage(final AuthenticatedBackupUser backupUser) {
return dynamoClient.getItem(GetItemRequest.builder()
.tableName(backupTableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId(backupUser))))
.projectionExpression("#mediaBytesUsed,#mediaCount,#usageRecalc")
.expressionAttributeNames(Map.of(
"#mediaBytesUsed", ATTR_MEDIA_BYTES_USED,
"#mediaCount", ATTR_MEDIA_COUNT,
"#usageRecalc", ATTR_MEDIA_USAGE_LAST_RECALCULATION))
.consistentRead(true)
.build())
.thenApply(response -> {
final long mediaUsed = AttributeValues.getLong(response.item(), ATTR_MEDIA_BYTES_USED, 0L);
final long mediaCount = AttributeValues.getLong(response.item(), ATTR_MEDIA_COUNT, 0L);
final long recalcSeconds = AttributeValues.getLong(response.item(), ATTR_MEDIA_USAGE_LAST_RECALCULATION, 0L);
return new TimestampedUsageInfo(new UsageInfo(mediaUsed, mediaCount), Instant.ofEpochSecond(recalcSeconds));
});
}
CompletableFuture<Void> setMediaUsage(final AuthenticatedBackupUser backupUser, UsageInfo usageInfo) {
return setMediaUsage(UpdateBuilder.forUser(backupTableName, backupUser), usageInfo);
}
CompletableFuture<Void> setMediaUsage(final StoredBackupAttributes backupAttributes, UsageInfo usageInfo) {
return setMediaUsage(new UpdateBuilder(backupTableName, BackupLevel.PAID, backupAttributes.hashedBackupId()), usageInfo);
}
private CompletableFuture<Void> setMediaUsage(final UpdateBuilder updateBuilder, UsageInfo usageInfo) {
return dynamoClient.updateItem(
updateBuilder
.addSetExpression("#mediaBytesUsed = :mediaBytesUsed",
Map.entry("#mediaBytesUsed", ATTR_MEDIA_BYTES_USED),
Map.entry(":mediaBytesUsed", AttributeValues.n(usageInfo.bytesUsed())))
.addSetExpression("#mediaCount = :mediaCount",
Map.entry("#mediaCount", ATTR_MEDIA_COUNT),
Map.entry(":mediaCount", AttributeValues.n(usageInfo.numObjects())))
.addSetExpression("#mediaRecalc = :mediaRecalc",
Map.entry("#mediaRecalc", ATTR_MEDIA_USAGE_LAST_RECALCULATION),
Map.entry(":mediaRecalc", AttributeValues.n(clock.instant().getEpochSecond())))
.updateItemBuilder()
.build())
.thenRun(Util.NOOP);
}
/**
* Marks the backup as undergoing expiration.
* <p>
* This must be called before beginning to delete items in the CDN with the prefix specified by
* {@link ExpiredBackup#prefixToDelete()}. If the prefix has been successfully deleted, {@link #finishExpiration} must
* be called.
*
* @param expiredBackup The backup to expire
* @return A stage that completes when the backup has been marked for expiration
*/
CompletableFuture<Void> startExpiration(final ExpiredBackup expiredBackup) {
if (expiredBackup.expirationType() == ExpiredBackup.ExpirationType.GARBAGE_COLLECTION) {
// We've already updated the row on a prior (failed) attempt, just need to remove the data from the cdn now
return CompletableFuture.completedFuture(null);
}
// Clear usage metadata, swap names of things we intend to delete, and record our intent to delete in attr_expired_prefix
return dynamoClient.updateItem(new UpdateBuilder(backupTableName, BackupLevel.PAID, expiredBackup.hashedBackupId())
.clearMediaUsage(clock)
.expireDirectoryNames(secureRandom, expiredBackup.expirationType())
.addRemoveExpression(Map.entry("#mediaRefresh", ATTR_LAST_MEDIA_REFRESH))
.addSetExpression("#expiredPrefix = :expiredPrefix",
Map.entry("#expiredPrefix", ATTR_EXPIRED_PREFIX),
Map.entry(":expiredPrefix", AttributeValues.s(expiredBackup.prefixToDelete())))
.withConditionExpression("attribute_not_exists(#expiredPrefix) OR #expiredPrefix = :expiredPrefix")
.updateItemBuilder()
.build())
.thenRun(Util.NOOP);
}
/**
* Complete expiration of a backup started with {@link #startExpiration}
* <p>
* If the expiration was for the entire backup, this will delete the entire item for the backup.
*
* @param expiredBackup The backup to expire
* @return A stage that completes when the expiration is marked as finished
*/
CompletableFuture<Void> finishExpiration(final ExpiredBackup expiredBackup) {
final byte[] hashedBackupId = expiredBackup.hashedBackupId();
if (expiredBackup.expirationType() == ExpiredBackup.ExpirationType.ALL) {
final long expectedLastRefresh = expiredBackup.lastRefresh().getEpochSecond();
return dynamoClient.deleteItem(DeleteItemRequest.builder()
.tableName(backupTableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId)))
.conditionExpression("#lastRefresh <= :expectedLastRefresh")
.expressionAttributeNames(Map.of("#lastRefresh", ATTR_LAST_REFRESH))
.expressionAttributeValues(Map.of(":expectedLastRefresh", AttributeValues.n(expectedLastRefresh)))
.build())
.thenRun(Util.NOOP);
} else {
return dynamoClient.updateItem(new UpdateBuilder(backupTableName, BackupLevel.PAID, hashedBackupId)
.addRemoveExpression(Map.entry("#expiredPrefixes", ATTR_EXPIRED_PREFIX))
.updateItemBuilder()
.build())
.thenRun(Util.NOOP);
}
}
Flux<StoredBackupAttributes> listBackupAttributes(final int segments) {
if (segments < 1) {
throw new IllegalArgumentException("Total number of segments must be positive");
}
return Flux.range(0, segments)
.flatMap(segment -> dynamoClient.scanPaginator(ScanRequest.builder()
.tableName(backupTableName)
.consistentRead(true)
.segment(segment)
.totalSegments(segments)
.expressionAttributeNames(Map.of(
"#backupIdHash", KEY_BACKUP_ID_HASH,
"#refresh", ATTR_LAST_REFRESH,
"#mediaRefresh", ATTR_LAST_MEDIA_REFRESH,
"#bytesUsed", ATTR_MEDIA_BYTES_USED,
"#numObjects", ATTR_MEDIA_COUNT,
"#backupDir", ATTR_BACKUP_DIR,
"#mediaDir", ATTR_MEDIA_DIR))
.projectionExpression("#backupIdHash, #refresh, #mediaRefresh, #bytesUsed, #numObjects, #backupDir, #mediaDir")
.build()))
// Don't use the SDK's item publisher, works around https://github.com/aws/aws-sdk-java-v2/issues/6411
.concatMap(page -> Flux.fromIterable(page.items()))
.filter(item -> item.containsKey(KEY_BACKUP_ID_HASH))
.map(BackupsDb::fromItem);
}
private static StoredBackupAttributes fromItem(Map<String, AttributeValue> item) {
return new StoredBackupAttributes(
AttributeValues.getByteArray(item, KEY_BACKUP_ID_HASH, null),
AttributeValues.getString(item, ATTR_BACKUP_DIR, null),
AttributeValues.getString(item, ATTR_MEDIA_DIR, null),
Instant.ofEpochSecond(AttributeValues.getLong(item, ATTR_LAST_REFRESH, 0L)),
Instant.ofEpochSecond(AttributeValues.getLong(item, ATTR_LAST_MEDIA_REFRESH, 0L)),
AttributeValues.getLong(item, ATTR_MEDIA_BYTES_USED, 0L),
AttributeValues.getLong(item, ATTR_MEDIA_COUNT, 0L));
}
Flux<ExpiredBackup> getExpiredBackups(final int segments, final Scheduler scheduler, final Instant purgeTime) {
if (segments < 1) {
throw new IllegalArgumentException("Total number of segments must be positive");
}
return Flux.range(0, segments)
.parallel()
.runOn(scheduler)
.flatMap(segment -> dynamoClient.scanPaginator(ScanRequest.builder()
.tableName(backupTableName)
.consistentRead(true)
.segment(segment)
.totalSegments(segments)
.expressionAttributeNames(Map.of(
"#backupIdHash", KEY_BACKUP_ID_HASH,
"#refresh", ATTR_LAST_REFRESH,
"#mediaRefresh", ATTR_LAST_MEDIA_REFRESH,
"#backupDir", ATTR_BACKUP_DIR,
"#mediaDir", ATTR_MEDIA_DIR,
"#expiredPrefix", ATTR_EXPIRED_PREFIX))
.expressionAttributeValues(Map.of(":purgeTime", AttributeValues.n(purgeTime.getEpochSecond())))
.projectionExpression("#backupIdHash, #refresh, #mediaRefresh, #backupDir, #mediaDir, #expiredPrefix")
.filterExpression(
"(#refresh < :purgeTime) OR (#mediaRefresh < :purgeTime) OR attribute_exists(#expiredPrefix)")
.build())
.items())
.sequential()
.filter(Predicate.not(Map::isEmpty))
.mapNotNull(item -> {
final byte[] hashedBackupId = AttributeValues.getByteArray(item, KEY_BACKUP_ID_HASH, null);
if (hashedBackupId == null) {
return null;
}
final String backupDir = AttributeValues.getString(item, ATTR_BACKUP_DIR, null);
final String mediaDir = AttributeValues.getString(item, ATTR_MEDIA_DIR, null);
if (backupDir == null || mediaDir == null) {
// Could be the case for backups that have not yet set a public key
return null;
}
final long lastRefresh = AttributeValues.getLong(item, ATTR_LAST_REFRESH, Long.MAX_VALUE);
final long lastMediaRefresh = AttributeValues.getLong(item, ATTR_LAST_MEDIA_REFRESH, Long.MAX_VALUE);
final String existingExpiration = AttributeValues.getString(item, ATTR_EXPIRED_PREFIX, null);
final ExpiredBackup expiredBackup;
if (existingExpiration != null) {
// If we have work from a failed previous expiration, handle that before worrying about any new expirations.
// This guarantees we won't accumulate expirations
expiredBackup = new ExpiredBackup(hashedBackupId, ExpiredBackup.ExpirationType.GARBAGE_COLLECTION,
Instant.ofEpochSecond(lastRefresh), existingExpiration);
} else if (lastRefresh < purgeTime.getEpochSecond()) {
// The whole backup was expired
expiredBackup = new ExpiredBackup(hashedBackupId, ExpiredBackup.ExpirationType.ALL,
Instant.ofEpochSecond(lastRefresh), backupDir);
} else if (lastMediaRefresh < purgeTime.getEpochSecond()) {
// The media was expired
expiredBackup = new ExpiredBackup(hashedBackupId, ExpiredBackup.ExpirationType.MEDIA,
Instant.ofEpochSecond(lastRefresh), backupDir + "/" + mediaDir);
} else {
return null;
}
if (!isValid(expiredBackup)) {
logger.error("Not expiring backup {} for backupId {} with invalid cdn path prefixes",
HexFormat.of().formatHex(expiredBackup.hashedBackupId()),
expiredBackup);
return null;
}
return expiredBackup;
});
}
/**
* Backup expiration will expire any prefix we tell it to, so confirm that the directory names that came out of the
* database have the correct shape before handing them off.
*
* @param expiredBackup The ExpiredBackup object to check
* @return Whether this is a valid expiration object
*/
private static boolean isValid(final ExpiredBackup expiredBackup) {
// expired prefixes should be of the form "backupDir" or "backupDir/mediaDir"
return switch (expiredBackup.expirationType()) {
case MEDIA -> expiredBackup.prefixToDelete().length() == MEDIA_DIRECTORY_PATH_LENGTH;
case ALL -> expiredBackup.prefixToDelete().length() == BACKUP_DIRECTORY_PATH_LENGTH;
case GARBAGE_COLLECTION -> expiredBackup.prefixToDelete().length() == MEDIA_DIRECTORY_PATH_LENGTH ||
expiredBackup.prefixToDelete().length() == BACKUP_DIRECTORY_PATH_LENGTH;
};
}
/**
* Build ddb update statements for the backups table
*/
private static class UpdateBuilder {
private final List<String> setStatements = new ArrayList<>();
private final List<String> removeStatements = new ArrayList<>();
private final Map<String, AttributeValue> attrValues = new HashMap<>();
private final Map<String, String> attrNames = new HashMap<>();
private final String tableName;
private final BackupLevel backupLevel;
private final byte[] hashedBackupId;
private String conditionExpression = null;
static UpdateBuilder forUser(String tableName, AuthenticatedBackupUser backupUser) {
return new UpdateBuilder(tableName, backupUser.backupLevel(), hashedBackupId(backupUser));
}
UpdateBuilder(String tableName, BackupLevel backupLevel, byte[] hashedBackupId) {
this.tableName = tableName;
this.backupLevel = backupLevel;
this.hashedBackupId = hashedBackupId;
}
private void addAttrValue(Map.Entry<String, AttributeValue> attrValue) {
final AttributeValue old = attrValues.put(attrValue.getKey(), attrValue.getValue());
if (old != null && !old.equals(attrValue.getValue())) {
throw new IllegalArgumentException("duplicate attrValue key used for different values");
}
}
private void addAttrName(Map.Entry<String, String> attrName) {
final String oldName = attrNames.put(attrName.getKey(), attrName.getValue());
if (oldName != null && !oldName.equals(attrName.getValue())) {
throw new IllegalArgumentException("duplicate attrName key used for different attribute names");
}
}
private void addAttrs(final Map.Entry<String, String> attrName, final Map.Entry<String, AttributeValue> attrValue) {
addAttrName(attrName);
addAttrValue(attrValue);
}
UpdateBuilder addSetExpression(
final String update,
final Map.Entry<String, String> attrName,
final Map.Entry<String, AttributeValue> attrValue) {
setStatements.add(update);
addAttrs(attrName, attrValue);
return this;
}
UpdateBuilder addSetExpression(final String update) {
setStatements.add(update);
return this;
}
UpdateBuilder addRemoveExpression(final Map.Entry<String, String> attrName) {
addAttrName(attrName);
removeStatements.add(attrName.getKey());
return this;
}
UpdateBuilder withConditionExpression(final String conditionExpression) {
this.conditionExpression = conditionExpression;
return this;
}
UpdateBuilder withConditionExpression(
final String conditionExpression,
final Map.Entry<String, String> attrName,
final Map.Entry<String, AttributeValue> attrValue) {
this.addAttrs(attrName, attrValue);
this.conditionExpression = conditionExpression;
return this;
}
UpdateBuilder setCdn(final int cdn) {
return addSetExpression(
"#cdn = :cdn",
Map.entry("#cdn", ATTR_CDN),
Map.entry(":cdn", AttributeValues.n(cdn)));
}
UpdateBuilder incrementMediaCount(long delta) {
addAttrName(Map.entry("#mediaCount", ATTR_MEDIA_COUNT));
addAttrValue(Map.entry(":zero", AttributeValues.n(0)));
addAttrValue(Map.entry(":mediaCountDelta", AttributeValues.n(delta)));
addSetExpression("#mediaCount = if_not_exists(#mediaCount, :zero) + :mediaCountDelta");
return this;
}
UpdateBuilder incrementMediaBytes(long delta) {
addAttrName(Map.entry("#mediaBytes", ATTR_MEDIA_BYTES_USED));
addAttrValue(Map.entry(":zero", AttributeValues.n(0)));
addAttrValue(Map.entry(":mediaBytesDelta", AttributeValues.n(delta)));
addSetExpression("#mediaBytes = if_not_exists(#mediaBytes, :zero) + :mediaBytesDelta");
return this;
}
UpdateBuilder clearMediaUsage(final Clock clock) {
addSetExpression("#mediaBytesUsed = :mediaBytesUsed",
Map.entry("#mediaBytesUsed", ATTR_MEDIA_BYTES_USED),
Map.entry(":mediaBytesUsed", AttributeValues.n(0L)));
addSetExpression("#mediaCount = :mediaCount",
Map.entry("#mediaCount", ATTR_MEDIA_COUNT),
Map.entry(":mediaCount", AttributeValues.n(0L)));
addSetExpression("#mediaRecalc = :mediaRecalc",
Map.entry("#mediaRecalc", ATTR_MEDIA_USAGE_LAST_RECALCULATION),
Map.entry(":mediaRecalc", AttributeValues.n(clock.instant().getEpochSecond())));
return this;
}
UpdateBuilder setDirectoryNamesIfMissing(final SecureRandom secureRandom) {
final String backupDir = generateDirName(secureRandom);
final String mediaDir = generateDirName(secureRandom);
addSetExpression("#backupDir = if_not_exists(#backupDir, :backupDir)",
Map.entry("#backupDir", ATTR_BACKUP_DIR),
Map.entry(":backupDir", AttributeValues.s(backupDir)));
addSetExpression("#mediaDir = if_not_exists(#mediaDir, :mediaDir)",
Map.entry("#mediaDir", ATTR_MEDIA_DIR),
Map.entry(":mediaDir", AttributeValues.s(mediaDir)));
return this;
}
UpdateBuilder expireDirectoryNames(
final SecureRandom secureRandom,
final ExpiredBackup.ExpirationType expirationType) {
final String backupDir = generateDirName(secureRandom);
final String mediaDir = generateDirName(secureRandom);
return switch (expirationType) {
case GARBAGE_COLLECTION -> this;
case MEDIA -> this.addSetExpression("#mediaDir = :mediaDir",
Map.entry("#mediaDir", ATTR_MEDIA_DIR),
Map.entry(":mediaDir", AttributeValues.s(mediaDir)));
case ALL -> this
.addSetExpression("#mediaDir = :mediaDir",
Map.entry("#mediaDir", ATTR_MEDIA_DIR),
Map.entry(":mediaDir", AttributeValues.s(mediaDir)))
.addSetExpression("#backupDir = :backupDir",
Map.entry("#backupDir", ATTR_BACKUP_DIR),
Map.entry(":backupDir", AttributeValues.s(backupDir)));
};
}
UpdateBuilder setRefreshTimes(final Clock clock) {
return setRefreshTimes(clock.instant().truncatedTo(ChronoUnit.DAYS));
}
/**
* Set the lastRefresh time as part of the update
* <p>
* This always updates lastRefreshTime, and updates lastMediaRefreshTime if the backup user has the appropriate
* level.
*/
UpdateBuilder setRefreshTimes(final Instant refreshTime) {
if (!refreshTime.truncatedTo(ChronoUnit.DAYS).equals(refreshTime)) {
throw new IllegalArgumentException("Refresh time must be day aligned");
}
addSetExpression("#lastRefreshTime = :lastRefreshTime",
Map.entry("#lastRefreshTime", ATTR_LAST_REFRESH),
Map.entry(":lastRefreshTime", AttributeValues.n(refreshTime.getEpochSecond())));
if (backupLevel.compareTo(BackupLevel.PAID) >= 0) {
// update the media time if we have the appropriate level
addSetExpression("#lastMediaRefreshTime = :lastMediaRefreshTime",
Map.entry("#lastMediaRefreshTime", ATTR_LAST_MEDIA_REFRESH),
Map.entry(":lastMediaRefreshTime", AttributeValues.n(refreshTime.getEpochSecond())));
}
return this;
}
private String updateExpression() {
final StringBuilder sb = new StringBuilder();
if (!setStatements.isEmpty()) {
sb.append("SET ");
sb.append(String.join(",", setStatements));
}
if (!removeStatements.isEmpty()) {
sb.append(" REMOVE ");
sb.append(String.join(",", removeStatements));
}
return sb.toString();
}
/**
* Prepare a non-transactional update
*
* @return An {@link UpdateItemRequest#builder()} that can be used with updateItem
*/
UpdateItemRequest.Builder updateItemBuilder() {
final UpdateItemRequest.Builder bldr = UpdateItemRequest.builder()
.tableName(tableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId)))
.updateExpression(updateExpression())
.expressionAttributeNames(attrNames);
if (!this.attrValues.isEmpty()) {
bldr.expressionAttributeValues(attrValues);
}
if (this.conditionExpression != null) {
bldr.conditionExpression(conditionExpression);
}
return bldr;
}
/**
* Prepare a transactional update
*
* @return An {@link Update#builder()} that can be used with transactItem
*/
Update.Builder transactItemBuilder() {
final Update.Builder bldr = Update.builder()
.tableName(tableName)
.key(Map.of(KEY_BACKUP_ID_HASH, AttributeValues.b(hashedBackupId)))
.updateExpression(updateExpression())
.expressionAttributeNames(attrNames)
.expressionAttributeValues(attrValues);
if (this.conditionExpression != null) {
bldr.conditionExpression(conditionExpression);
}
return bldr;
}
}
static String generateDirName(final SecureRandom secureRandom) {
final byte[] bytes = new byte[16];
secureRandom.nextBytes(bytes);
return Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
}
private static byte[] hashedBackupId(final AuthenticatedBackupUser backupId) {
return hashedBackupId(backupId.backupId());
}
static byte[] hashedBackupId(final byte[] backupId) {
try {
return Arrays.copyOf(MessageDigest.getInstance("SHA-256").digest(backupId), 16);
} catch (NoSuchAlgorithmException e) {
throw new AssertionError(e);
}
}
}

View File

@@ -0,0 +1,78 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import org.apache.http.HttpHeaders;
import org.whispersystems.textsecuregcm.attachments.TusConfiguration;
import org.whispersystems.textsecuregcm.auth.ExternalServiceCredentials;
import org.whispersystems.textsecuregcm.auth.ExternalServiceCredentialsGenerator;
import org.whispersystems.textsecuregcm.util.HeaderUtils;
import java.nio.charset.StandardCharsets;
import java.time.Clock;
import java.util.Base64;
import java.util.Map;
public class Cdn3BackupCredentialGenerator {
public static final String CDN_PATH = "backups";
public static final int BACKUP_CDN = 3;
private static String READ_PERMISSION = "read";
private static String WRITE_PERMISSION = "write";
private static String PERMISSION_SEPARATOR = "$";
// Write entities will be of the form 'write$backups/<string>
private static final String WRITE_ENTITY_PREFIX = String.format("%s%s%s/", WRITE_PERMISSION, PERMISSION_SEPARATOR,
CDN_PATH);
// Read entities will be of the form 'read$backups/<string>
private static final String READ_ENTITY_PREFIX = String.format("%s%s%s/", READ_PERMISSION, PERMISSION_SEPARATOR,
CDN_PATH);
private final ExternalServiceCredentialsGenerator credentialsGenerator;
private final String tusUri;
public Cdn3BackupCredentialGenerator(final TusConfiguration cfg) {
this.tusUri = cfg.uploadUri();
this.credentialsGenerator = credentialsGenerator(Clock.systemUTC(), cfg);
}
private static ExternalServiceCredentialsGenerator credentialsGenerator(final Clock clock,
final TusConfiguration cfg) {
return ExternalServiceCredentialsGenerator
.builder(cfg.userAuthenticationTokenSharedSecret())
.prependUsername(false)
.withClock(clock)
.build();
}
public BackupUploadDescriptor generateUpload(final String key) {
if (key.isBlank()) {
throw new IllegalArgumentException("Upload descriptors must have non-empty keys");
}
final String entity = WRITE_ENTITY_PREFIX + key;
final ExternalServiceCredentials credentials = credentialsGenerator.generateFor(entity);
final String b64Key = Base64.getEncoder().encodeToString(key.getBytes(StandardCharsets.UTF_8));
final Map<String, String> headers = Map.of(
HttpHeaders.AUTHORIZATION, HeaderUtils.basicAuthHeader(credentials),
"Upload-Metadata", String.format("filename %s", b64Key));
return new BackupUploadDescriptor(
BACKUP_CDN,
key,
headers,
tusUri + "/" + CDN_PATH);
}
public Map<String, String> readHeaders(final String hashedBackupId) {
if (hashedBackupId.isBlank()) {
throw new IllegalArgumentException("Backup subdir name must be non-empty");
}
final ExternalServiceCredentials credentials = credentialsGenerator.generateFor(
READ_ENTITY_PREFIX + hashedBackupId);
return Map.of(HttpHeaders.AUTHORIZATION, HeaderUtils.basicAuthHeader(credentials));
}
}

View File

@@ -0,0 +1,313 @@
package org.whispersystems.textsecuregcm.backup;
import com.fasterxml.jackson.core.JsonProcessingException;
import io.micrometer.core.instrument.Metrics;
import io.micrometer.core.instrument.Timer;
import jakarta.validation.constraints.NotNull;
import jakarta.ws.rs.core.Response;
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ScheduledExecutorService;
import javax.annotation.Nullable;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.whispersystems.textsecuregcm.configuration.Cdn3StorageManagerConfiguration;
import org.whispersystems.textsecuregcm.http.FaultTolerantHttpClient;
import org.whispersystems.textsecuregcm.metrics.MetricsUtil;
import org.whispersystems.textsecuregcm.util.ExceptionUtils;
import org.whispersystems.textsecuregcm.util.HttpUtils;
import org.whispersystems.textsecuregcm.util.SystemMapper;
public class Cdn3RemoteStorageManager implements RemoteStorageManager {
private static final Logger logger = LoggerFactory.getLogger(Cdn3RemoteStorageManager.class);
private final FaultTolerantHttpClient storageManagerHttpClient;
private final String storageManagerBaseUrl;
private final String clientId;
private final String clientSecret;
private final Map<Integer, String> sourceSchemes;
static final String CLIENT_ID_HEADER = "CF-Access-Client-Id";
static final String CLIENT_SECRET_HEADER = "CF-Access-Client-Secret";
private static final String STORAGE_MANAGER_STATUS_COUNTER_NAME = MetricsUtil.name(Cdn3RemoteStorageManager.class,
"storageManagerStatus");
private static final String STORAGE_MANAGER_TIMER_NAME = MetricsUtil.name(Cdn3RemoteStorageManager.class,
"storageManager");
private static final String OPERATION_TAG_NAME = "op";
private static final String STATUS_TAG_NAME = "status";
private static final String OBJECT_REMOVED_ON_DELETE_COUNTER_NAME = MetricsUtil.name(Cdn3RemoteStorageManager.class, "objectRemovedOnDelete");
public Cdn3RemoteStorageManager(
final ExecutorService httpExecutor,
final ScheduledExecutorService retryExecutor,
final Cdn3StorageManagerConfiguration configuration) {
// strip trailing "/" for easier URI construction
this.storageManagerBaseUrl = StringUtils.removeEnd(configuration.baseUri(), "/");
this.clientId = configuration.clientId();
this.clientSecret = configuration.clientSecret().value();
// Client used for calls to storage-manager
this.storageManagerHttpClient = FaultTolerantHttpClient.newBuilder("cdn3-storage-manager", httpExecutor)
.withCircuitBreaker(configuration.circuitBreakerConfigurationName())
.withRetry(configuration.retryConfigurationName(), retryExecutor)
.withConnectTimeout(Duration.ofSeconds(10))
.withVersion(HttpClient.Version.HTTP_2)
.withNumClients(configuration.numHttpClients())
.build();
this.sourceSchemes = configuration.sourceSchemes();
}
@Override
public int cdnNumber() {
return 3;
}
@Override
public CompletionStage<Void> copy(
final int sourceCdn,
final String sourceKey,
final int expectedSourceLength,
final MediaEncryptionParameters encryptionParameters,
final String destinationKey) {
final String sourceScheme = this.sourceSchemes.get(sourceCdn);
if (sourceScheme == null) {
return CompletableFuture.failedFuture(
new SourceObjectNotFoundException("Cdn3RemoteStorageManager cannot copy from " + sourceCdn));
}
final String requestBody = new Cdn3CopyRequest(
encryptionParameters,
new Cdn3CopyRequest.SourceDescriptor(sourceScheme, sourceKey),
expectedSourceLength,
destinationKey).json();
final Timer.Sample sample = Timer.start();
final HttpRequest request = HttpRequest.newBuilder()
.PUT(HttpRequest.BodyPublishers.ofString(requestBody))
.uri(URI.create(copyUrl()))
.header("Content-Type", "application/json")
.header(CLIENT_ID_HEADER, clientId)
.header(CLIENT_SECRET_HEADER, clientSecret)
.build();
return this.storageManagerHttpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenAccept(response -> {
Metrics.counter(STORAGE_MANAGER_STATUS_COUNTER_NAME,
OPERATION_TAG_NAME, "copy",
STATUS_TAG_NAME, Integer.toString(response.statusCode()))
.increment();
if (response.statusCode() == Response.Status.NOT_FOUND.getStatusCode()) {
throw ExceptionUtils.wrap(new SourceObjectNotFoundException());
} else if (response.statusCode() == Response.Status.CONFLICT.getStatusCode()) {
throw ExceptionUtils.wrap(new InvalidLengthException(response.body()));
} else if (!HttpUtils.isSuccessfulResponse(response.statusCode())) {
logger.info("Failed to copy via storage-manager {} {}", response.statusCode(), response.body());
throw ExceptionUtils.wrap(new IOException("Failed to copy object: " + response.statusCode()));
}
})
.whenComplete((ignored, ignoredException) ->
sample.stop(Metrics.timer(STORAGE_MANAGER_TIMER_NAME, OPERATION_TAG_NAME, "copy")));
}
/**
* Serialized copy request for cdn3 storage manager
*/
record Cdn3CopyRequest(
String encryptionKey, String hmacKey,
SourceDescriptor source, int expectedSourceLength,
String dst) {
Cdn3CopyRequest(MediaEncryptionParameters parameters, SourceDescriptor source, int expectedSourceLength,
String dst) {
this(Base64.getEncoder().encodeToString(parameters.aesEncryptionKey().getEncoded()),
Base64.getEncoder().encodeToString(parameters.hmacSHA256Key().getEncoded()),
source, expectedSourceLength, dst);
}
record SourceDescriptor(String scheme, String key) {}
String json() {
try {
return SystemMapper.jsonMapper().writeValueAsString(this);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Could not serialize copy request", e);
}
}
}
@Override
public CompletionStage<ListResult> list(
final String prefix,
final Optional<String> cursor,
final long limit) {
final Timer.Sample sample = Timer.start();
final Map<String, String> queryParams = new HashMap<>();
queryParams.put("prefix", prefix);
queryParams.put("limit", Long.toString(limit));
cursor.ifPresent(s -> queryParams.put("cursor", cursor.get()));
final HttpRequest request = HttpRequest.newBuilder().GET()
.uri(URI.create("%s%s".formatted(listUrl(), HttpUtils.queryParamString(queryParams.entrySet()))))
.header(CLIENT_ID_HEADER, clientId)
.header(CLIENT_SECRET_HEADER, clientSecret)
.build();
return this.storageManagerHttpClient.sendAsync(request, HttpResponse.BodyHandlers.ofInputStream())
.thenApply(response -> {
Metrics.counter(STORAGE_MANAGER_STATUS_COUNTER_NAME,
OPERATION_TAG_NAME, "list",
STATUS_TAG_NAME, Integer.toString(response.statusCode()))
.increment();
try {
return parseListResponse(response, prefix);
} catch (IOException e) {
throw ExceptionUtils.wrap(e);
}
})
.whenComplete((ignored, ignoredException) ->
sample.stop(Metrics.timer(STORAGE_MANAGER_TIMER_NAME, OPERATION_TAG_NAME, "list")));
}
/**
* Serialized list response from storage manager
*/
record Cdn3ListResponse(@NotNull List<Entry> objects, @Nullable String cursor) {
record Entry(@NotNull String key, @NotNull long size) {}
}
private static ListResult parseListResponse(final HttpResponse<InputStream> httpListResponse, final String prefix)
throws IOException {
if (!HttpUtils.isSuccessfulResponse(httpListResponse.statusCode())) {
throw new IOException("Failed to list objects: " + httpListResponse.statusCode());
}
final Cdn3ListResponse result = SystemMapper.jsonMapper()
.readValue(httpListResponse.body(), Cdn3ListResponse.class);
final List<ListResult.Entry> objects = new ArrayList<>(result.objects.size());
for (Cdn3ListResponse.Entry entry : result.objects) {
if (!entry.key().startsWith(prefix)) {
logger.error("unexpected listing result from cdn3 - entry {} does not contain requested prefix {}",
entry.key(), prefix);
throw new IOException("prefix listing returned unexpected result");
}
objects.add(new ListResult.Entry(entry.key().substring(prefix.length()), entry.size()));
}
return new ListResult(objects, Optional.ofNullable(result.cursor));
}
/**
* Serialized usage response from storage manager
*/
record UsageResponse(@NotNull long numObjects, @NotNull long bytesUsed) {}
@Override
public CompletionStage<UsageInfo> calculateBytesUsed(final String prefix) {
final Timer.Sample sample = Timer.start();
final HttpRequest request = HttpRequest.newBuilder().GET()
.uri(URI.create("%s%s".formatted(
usageUrl(),
HttpUtils.queryParamString(Map.of("prefix", prefix).entrySet()))))
.header(CLIENT_ID_HEADER, clientId)
.header(CLIENT_SECRET_HEADER, clientSecret)
.build();
return this.storageManagerHttpClient.sendAsync(request, HttpResponse.BodyHandlers.ofInputStream())
.thenApply(response -> {
Metrics.counter(STORAGE_MANAGER_STATUS_COUNTER_NAME,
OPERATION_TAG_NAME, "usage",
STATUS_TAG_NAME, Integer.toString(response.statusCode()))
.increment();
try {
return parseUsageResponse(response);
} catch (IOException e) {
throw ExceptionUtils.wrap(e);
}
})
.whenComplete((ignored, ignoredException) ->
sample.stop(Metrics.timer(STORAGE_MANAGER_TIMER_NAME, OPERATION_TAG_NAME, "usage")));
}
private static UsageInfo parseUsageResponse(final HttpResponse<InputStream> httpUsageResponse) throws IOException {
if (!HttpUtils.isSuccessfulResponse(httpUsageResponse.statusCode())) {
throw new IOException("Failed to retrieve usage: " + httpUsageResponse.statusCode());
}
final UsageResponse response = SystemMapper.jsonMapper().readValue(httpUsageResponse.body(), UsageResponse.class);
return new UsageInfo(response.bytesUsed(), response.numObjects);
}
/**
* Serialized delete response from storage manager
*/
record DeleteResponse(@NotNull long bytesDeleted) {}
public CompletionStage<Long> delete(final String key) {
final Timer.Sample sample = Timer.start();
final HttpRequest request = HttpRequest.newBuilder().DELETE()
.uri(URI.create(deleteUrl(key)))
.header(CLIENT_ID_HEADER, clientId)
.header(CLIENT_SECRET_HEADER, clientSecret)
.build();
return this.storageManagerHttpClient.sendAsync(request, HttpResponse.BodyHandlers.ofInputStream())
.thenApply(response -> {
Metrics.counter(STORAGE_MANAGER_STATUS_COUNTER_NAME,
OPERATION_TAG_NAME, "delete",
STATUS_TAG_NAME, Integer.toString(response.statusCode()))
.increment();
try {
long bytesDeleted = parseDeleteResponse(response);
Metrics.counter(OBJECT_REMOVED_ON_DELETE_COUNTER_NAME,
"removed", Boolean.toString(bytesDeleted > 0))
.increment();
return bytesDeleted;
} catch (IOException e) {
throw ExceptionUtils.wrap(e);
}
})
.whenComplete((ignored, ignoredException) ->
sample.stop(Metrics.timer(STORAGE_MANAGER_TIMER_NAME, OPERATION_TAG_NAME, "delete")));
}
private long parseDeleteResponse(final HttpResponse<InputStream> httpDeleteResponse) throws IOException {
if (!HttpUtils.isSuccessfulResponse(httpDeleteResponse.statusCode())) {
throw new IOException("Failed to retrieve usage: " + httpDeleteResponse.statusCode());
}
return SystemMapper.jsonMapper().readValue(httpDeleteResponse.body(), DeleteResponse.class).bytesDeleted();
}
private String deleteUrl(final String key) {
return "%s/%s/%s".formatted(storageManagerBaseUrl, Cdn3BackupCredentialGenerator.CDN_PATH, key);
}
private String usageUrl() {
return "%s/usage".formatted(storageManagerBaseUrl);
}
private String listUrl() {
return "%s/%s/".formatted(storageManagerBaseUrl, Cdn3BackupCredentialGenerator.CDN_PATH);
}
private String copyUrl() {
return "%s/copy".formatted(storageManagerBaseUrl);
}
}

View File

@@ -0,0 +1,29 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
/**
* Descriptor for a single copy-and-encrypt operation
*
* @param sourceCdn The cdn of the object to copy
* @param sourceKey The mediaId within the cdn of the object to copy
* @param sourceLength The length of the object to copy
* @param encryptionParameters Encryption parameters to double encrypt the object
* @param destinationMediaId The mediaId of the destination object
*/
public record CopyParameters(
int sourceCdn,
String sourceKey,
int sourceLength,
MediaEncryptionParameters encryptionParameters,
byte[] destinationMediaId) {
/**
* @return The size of the double-encrypted destination object after it is copied
*/
long destinationObjectSize() {
return encryptionParameters().outputSize(sourceLength());
}
}

View File

@@ -0,0 +1,46 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import org.whispersystems.textsecuregcm.util.ExceptionUtils;
import javax.annotation.Nullable;
import java.util.Optional;
/**
* The result of a copy operation
*
* @param outcome Whether the copy was a success
* @param mediaId The destination mediaId
* @param cdn On success, the destination cdn
*/
public record CopyResult(Outcome outcome, byte[] mediaId, @Nullable Integer cdn) {
public enum Outcome {
SUCCESS,
SOURCE_NOT_FOUND,
SOURCE_WRONG_LENGTH,
OUT_OF_QUOTA
}
/**
* Map an exception returned by {@link RemoteStorageManager#copy} to CopyResult with the appropriate outcome.
*
* @param throwable result of a failed copy operation
* @param key the copy destination mediaId
* @return The appropriate CopyResult, or empty if the exception does not match to an Outcome.
*/
static Optional<CopyResult> fromCopyError(final Throwable throwable, final byte[] key) {
final Throwable unwrapped = ExceptionUtils.unwrap(throwable);
if (unwrapped instanceof SourceObjectNotFoundException) {
return Optional.of(new CopyResult(Outcome.SOURCE_NOT_FOUND, key, null));
} else if (unwrapped instanceof InvalidLengthException) {
return Optional.of(new CopyResult(Outcome.SOURCE_WRONG_LENGTH, key, null));
} else {
return Optional.empty();
}
}
}

View File

@@ -0,0 +1,31 @@
/*
* Copyright 2024 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import java.time.Instant;
/**
* Represents a backup that requires some or all of its content to be deleted
*
* @param hashedBackupId The hashedBackupId that owns this content
* @param expirationType What triggered the expiration
* @param lastRefresh The timestamp of the last time the backup user was seen
* @param prefixToDelete The prefix on the CDN associated with this backup that should be deleted
*/
public record ExpiredBackup(
byte[] hashedBackupId,
ExpirationType expirationType,
Instant lastRefresh,
String prefixToDelete) {
public enum ExpirationType {
// The prefixToDelete expiration is for the entire backup
ALL,
// The prefixToDelete is for the media associated with the backup
MEDIA,
// The prefixToDelete is from a prior expiration attempt
GARBAGE_COLLECTION
}
}

View File

@@ -0,0 +1,10 @@
package org.whispersystems.textsecuregcm.backup;
import java.io.IOException;
public class InvalidLengthException extends IOException {
public InvalidLengthException(String s) {
super(s);
}
}

View File

@@ -0,0 +1,22 @@
package org.whispersystems.textsecuregcm.backup;
import javax.crypto.spec.SecretKeySpec;
public record MediaEncryptionParameters(
SecretKeySpec aesEncryptionKey,
SecretKeySpec hmacSHA256Key) {
public MediaEncryptionParameters(byte[] encryptionKey, byte[] macKey) {
this(
new SecretKeySpec(encryptionKey, "AES"),
new SecretKeySpec(macKey, "HmacSHA256"));
}
public int outputSize(final int inputSize) {
// AES-256 has 16-byte block size, and always adds a block if the plaintext is a multiple of the block size
final int numBlocks = (inputSize + 16) / 16;
// 16-byte IV will be generated and prepended to the ciphertext
// IV + AES-256 encrypted data + HmacSHA256
return 16 + (numBlocks * 16) + 32;
}
}

View File

@@ -0,0 +1,6 @@
package org.whispersystems.textsecuregcm.backup;
import java.io.IOException;
public class PublicKeyConflictException extends IOException {
}

View File

@@ -0,0 +1,86 @@
package org.whispersystems.textsecuregcm.backup;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.CompletionStage;
/**
* Handles management operations over a external cdn storage system.
*/
public interface RemoteStorageManager {
/**
* @return The cdn number that this RemoteStorageManager manages
*/
int cdnNumber();
/**
* Copy and the object from a remote source into the backup, adding an additional layer of encryption
*
* @param sourceCdn The cdn number where the source attachment is stored
* @param sourceKey The key of the source attachment within the attachment cdn
* @param expectedSourceLength The length of the source object, should match the content-length of the object returned
* from the sourceUri.
* @param encryptionParameters The encryption keys that should be used to apply an additional layer of encryption to
* the object
* @param dstKey The key within the backup cdn where the copied object will be written
* @return A stage that completes successfully when the source has been successfully re-encrypted and copied into
* uploadDescriptor. The returned CompletionStage can be completed exceptionally with the following exceptions.
* <ul>
* <li> {@link InvalidLengthException} If the expectedSourceLength does not match the length of the sourceUri </li>
* <li> {@link SourceObjectNotFoundException} If the no object at sourceUri is found </li>
* <li> {@link java.io.IOException} If there was a generic IO issue </li>
* </ul>
*/
CompletionStage<Void> copy(
int sourceCdn,
String sourceKey,
int expectedSourceLength,
MediaEncryptionParameters encryptionParameters,
String dstKey);
/**
* Result of a {@link #list} operation
*
* @param objects An {@link Entry} for each object returned by the list request
* @param cursor An opaque string that can be used to resume listing from where a previous request left off, empty if
* the request reached the end of the list of matching objects.
*/
record ListResult(List<Entry> objects, Optional<String> cursor) {
/**
* An entry representing a remote stored object under a prefix
*
* @param key The name of the object with the prefix removed
* @param length The length of the object in bytes
*/
record Entry(String key, long length) {}
}
/**
* List objects on the remote cdn.
*
* @param prefix The prefix of the objects to list
* @param cursor The cursor returned by a previous call to list, or empty if starting from the first object with the
* provided prefix
* @param limit The maximum number of items to return in the list
* @return A {@link ListResult} of objects that match the prefix.
*/
CompletionStage<ListResult> list(final String prefix, final Optional<String> cursor, final long limit);
/**
* Calculate the total number of bytes stored by objects with the provided prefix
*
* @param prefix The prefix of the objects to sum
* @return The number of bytes used
*/
CompletionStage<UsageInfo> calculateBytesUsed(final String prefix);
/**
* Delete the specified object.
*
* @param key the key of the stored object to delete.
* @return the number of bytes freed by the deletion operation
*/
CompletionStage<Long> delete(final String key);
}

View File

@@ -0,0 +1,33 @@
/*
* Copyright 2025 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import com.google.common.annotations.VisibleForTesting;
import org.whispersystems.textsecuregcm.auth.ExternalServiceCredentialsGenerator;
import org.whispersystems.textsecuregcm.configuration.SecureValueRecoveryConfiguration;
import java.time.Clock;
public class SecureValueRecoveryBCredentialsGeneratorFactory {
private SecureValueRecoveryBCredentialsGeneratorFactory() {}
@VisibleForTesting
static ExternalServiceCredentialsGenerator svrbCredentialsGenerator(
final SecureValueRecoveryConfiguration cfg,
final Clock clock) {
return ExternalServiceCredentialsGenerator
.builder(cfg.userAuthenticationTokenSharedSecret())
.withUserDerivationKey(cfg.userIdTokenSharedSecret().value())
.prependUsername(false)
.withDerivedUsernameTruncateLength(16)
.withClock(clock)
.build();
}
public static ExternalServiceCredentialsGenerator svrbCredentialsGenerator(final SecureValueRecoveryConfiguration cfg) {
return svrbCredentialsGenerator(cfg, Clock.systemUTC());
}
}

View File

@@ -0,0 +1,17 @@
/*
* Copyright 2023 Signal Messenger, LLC
* SPDX-License-Identifier: AGPL-3.0-only
*/
package org.whispersystems.textsecuregcm.backup;
import java.io.IOException;
public class SourceObjectNotFoundException extends IOException {
public SourceObjectNotFoundException() {
super();
}
public SourceObjectNotFoundException(String message) {
super(message);
}
}

Some files were not shown because too many files have changed in this diff Show More