Commit Graph

770 Commits

Author SHA1 Message Date
Barış Soner Uşaklı
e145330c37 perf: convert expireAt index to partial
no need to store null in index for all documents
2026-04-03 20:17:06 -04:00
Barış Soner Uşaklı
2185f22a55 fix: try a save point in retry 2026-03-28 15:09:03 -04:00
Barış Soner Uşaklı
988f513655 fix: try upsert type if it fails 2026-03-28 14:32:04 -04:00
Barış Soner Uşaklı
10814b64a5 Merge branch 'master' into develop 2026-03-28 13:25:26 -04:00
Michele Di Maria
b8fd88fba9 Fix the saving of the statistics on PosgreSQL #14124 (#14129)
* fix: deduplicate postgres sorted set bulk ops to prevent pkey violation

sortedSetIncrByBulk and sortedSetAddBulk did not deduplicate (key, value)
pairs before INSERT, causing "duplicate key value violates unique constraint
legacy_zset_pkey" errors since PostgreSQL ON CONFLICT only resolves against
existing table rows, not within-statement duplicates.

Also adds missing pageviews:ap metrics to analyticsKeys sorted set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use upsert with RETURNING to prevent postgres analytics write failures

Replace the INSERT ON CONFLICT DO NOTHING + separate SELECT verification
pattern with INSERT ON CONFLICT DO UPDATE RETURNING. The old pattern had
an unreliable gap between INSERT and SELECT causing random "failed to
insert keys for objects" errors that blocked all analytics writes.

The no-op upsert (DO UPDATE SET type = existing type) guarantees every
row is returned via RETURNING, eliminating the need for a separate SELECT
and the "missing keys" check entirely. Also deduplicates the keys array
to prevent "cannot affect row a second time" errors with DO UPDATE.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 13:24:34 -04:00
Barış Soner Uşaklı
4b503db497 refactor: break long line 2026-03-27 16:44:10 -04:00
Barış Soner Uşaklı
28fafcdc56 retrt e11000 in incrObjectField 2026-03-27 11:48:45 -04:00
Barış Soner Uşaklı
837d984b84 refactor: some cleanup of dbal code 2026-03-27 11:24:43 -04:00
Barış Soner Uşaklı
202a99bb48 perf: use $or instead of double $in, in sortedSetIncrByBulk 2026-03-27 10:29:02 -04:00
Julian Lam
781ed3447b feat: track user cids (#14114)
* feat: start tracking which cids a user has posted to, update account pages' topics/posts view to call this new sorted set

re: #14113

* feat: upgrade script for #14113

* fix: cids unavailable in getPostsFields, duh

* fix: update sortedSetIncrByBulk in mongo/psql to return early on empty data

* fix: remove unused lodash require

* test: sortedSetIncrBy and sortedSetIncrByBulk tests

* test: who needs null checks anyway

* fix: sortedSetIncrByBulk null response

* test: aggregate zincrbulk data

if there are alot of identical key/value pairs they will be combined into a single row

* fix: key name

* test: fix test name

* lint: fix lint issues

* test: negative values should work too

* fix: add e11000 handler for incrByBulk

* refactor: fix variable name

* merge tests with existing zset test, remove dupes

* test: return topicData for failing test

* delete uid:<uid>:cids on user delete

---------

Co-authored-by: Barış Soner Uşaklı <barisusakli@gmail.com>
2026-03-26 10:30:28 -04:00
Barış Soner Uşaklı
09de6fb9ae perf: switch to set, remove parseFloat in redis
add test to cover float
2026-03-14 17:39:43 -04:00
Barış Uşaklı
79fe0902fe Partial set index (#14087)
* partial index test

* cleanup

* remove line

* background true

* whitespace

* test: fix invite/pending removal
2026-03-11 23:49:22 -04:00
Barış Soner Uşaklı
bcbb7bc444 perf: make a single round trip for set(s)Remove
use bulkWrite
2026-03-09 18:33:24 -04:00
Barış Soner Uşaklı
1c7daf0db9 refactor: switch to cursor 2026-03-05 20:31:36 -05:00
Barış Uşaklı
7d50baecf0 scan. glob to regex (#14063)
* scan. glob to regex

* lint
2026-03-05 16:01:03 -05:00
Barış Uşaklı
62d88555ae feat: eslint10 (#13967)
* feat: eslint10

* lint: no-useless-assignment
2026-02-10 12:39:11 -05:00
Barış Soner Uşaklı
2a5bd6ef36 Merge branch 'master' into develop 2025-12-24 23:27:06 -05:00
Barış Uşaklı
550411fb58 test: change redis connection (#13844) 2025-12-17 16:56:07 -05:00
Barış Uşaklı
6956270411 test: add a test for set db.exists (#13809)
* test: add a test for set db.exists

* delete empty sets

* test: psql
2025-12-04 18:33:55 -05:00
Barış Uşaklı
7d5402fe66 feat: setAddBulk (#13805)
* feat: setAddBulk

add some tests

* fix: sAdd with value array on redis
2025-12-03 18:18:14 -05:00
Barış Soner Uşaklı
70169758ec Merge branch 'master' into develop 2025-12-03 09:49:22 -05:00
Barış Soner Uşaklı
53e22acffb fix: remove hardcoded name for sentinel, #13794 2025-12-02 11:12:05 -05:00
Barış Uşaklı
287b25695d test: new mongodb deps (#13793) 2025-12-02 11:08:20 -05:00
Barış Soner Uşaklı
2142b680d9 chore: remove log 2025-12-01 17:47:46 -05:00
Barış Soner Uşaklı
5bd1f7b7ac feat: #13790, allow ssl setup in psql 2025-12-01 17:46:01 -05:00
Barış Soner Uşaklı
6cca55e37f fix: use parameterized query for key lookup 2025-09-17 10:50:35 -04:00
Barış Soner Uşaklı
f9ddbebacc fix: remove .auth call 2025-09-12 11:33:53 -04:00
Barış Soner Uşaklı
290a9395c0 fix: pass object to.auth 2025-09-06 13:47:46 -04:00
Barış Soner Uşaklı
09898b94ec fix: return null if field is falsy
fixes MongoServerError: FieldPath cannot be constructed with empty string error when getObjectField is called with a falsy value
2025-08-22 11:06:47 -04:00
Barış Soner Uşaklı
eac3d0a043 fix: redis connect host/port 2025-07-20 11:57:34 -04:00
Barış Soner Uşaklı
0b9bfc1ce1 refactor: parallel socket.io adapter 2025-06-18 16:59:57 -04:00
Barış Soner Uşaklı
f7f70468fd fix: pubsub on node-redis 2025-06-18 13:17:29 -04:00
Barış Uşaklı
14043ab0fd Node redis (#13500)
* refactor: start migrating to node-redis

* few more zset fixes

* fix: db.scan

* fix: list methods

* fix set methods

* fix: hash methods

* use hasOwn, remove cloning

* sorted set fixes

* fix: so data is converted to strings before saving

otherwise node-redis throws below error
TypeError: "arguments[2]" must be of type "string | Buffer", got number instead.

* chore: remove comments

* fix: zrank string param

* use new close

* chore: up dbsearch

* test: add log

* test: more log

* test: log failing test

* test: catch errors in formatApiResponse

add await so exception goes to catch

* tetst: add log

* fix: dont set null/undefined values

* test: more fixes
2025-06-18 13:04:57 -04:00
Barış Soner Uşaklı
16504bad81 fix: sql injection in sortedSetScan 2025-05-12 10:02:59 -04:00
Barış Uşaklı
b57ce29dc6 chore: up pg, pg-cursor (#13351)
* chore: up pg, pg-cursor

* test: check file directly
2025-04-22 13:32:41 -04:00
Barış Soner Uşaklı
4c92248a4b list remove all (#13113)
* list remove all

* one more test

* sortedSetIncrByBulk

* remove name

* incrObjectFieldByBulk

* test: disable api tests

* try merge

* another test

* give upon bulk incr

* update so answer

* one more try

* fix: name

* chore: up dbsearch
2025-02-02 03:30:52 -05:00
Barış Soner Uşaklı
0087764ce0 Merge branch 'master' into develop 2024-12-24 12:37:18 -05:00
Barış Soner Uşaklı
c60cee5997 fix: #13007, replace $ with \uFF04, like . 2024-12-24 12:37:06 -05:00
Barış Soner Uşaklı
eb49a64974 Merge branch 'master' into develop 2024-12-24 10:13:51 -05:00
Barış Soner Uşaklı
a6f0eba5a1 fix: test on redis 2024-12-23 20:39:52 -05:00
Barış Soner Uşaklı
c69fa811e5 fix: convert to str than filter
prevent empty strings from getting passed to mongodb
https://github.com/NodeBB/NodeBB/issues/13007#issuecomment-2560503754
2024-12-23 20:29:42 -05:00
Julian Lam
2ab72ada3f Merge remote-tracking branch 'origin/develop' into activitypub 2024-12-11 10:12:33 -05:00
Barış Soner Uşaklı
9d2af7e10c refactor: replace math.random with crypto 2024-12-09 18:18:02 -05:00
Julian Lam
45d2fe11dc Merge remote-tracking branch 'origin/develop' into activitypub 2024-12-04 15:24:26 -05:00
Barış Soner Uşaklı
8c5d7c4d13 deps: update connect-redis to 8 2024-12-01 09:29:46 -05:00
Barış Soner Uşaklı
c276aa1a99 Merge branch 'develop' into activitypub 2024-06-14 20:45:39 -04:00
Barış Soner Uşaklı
92f1f189a8 fix mongo stats 2024-06-14 20:23:50 -04:00
Barış Soner Uşaklı
6f831d1ce2 Merge branch 'develop' into activitypub 2024-06-08 12:36:51 -04:00
Barış Soner Uşaklı
1b283ccc29 test: return empty array on db.exists([]); 2024-06-07 21:44:57 -04:00
Barış Soner Uşaklı
b1f9ad5534 Squashed commit of the following:
commit 4e0e792232
Merge: 24d0999fb5 70b4a0e2ae
Author: Barış Soner Uşaklı <barisusakli@gmail.com>
Date:   Fri Jun 7 19:26:49 2024 -0400

    Merge branch 'master' into develop

commit 70b4a0e2ae
Author: Barış Soner Uşaklı <barisusakli@gmail.com>
Date:   Fri Jun 7 19:14:13 2024 -0400

    feat: allow passing min,max to sortedSetsCardSum

    to get rid of multiple db calls in profile page

commit 6bbe3d1c4c
Author: Barış Soner Uşaklı <barisusakli@gmail.com>
Date:   Fri Jun 7 14:08:48 2024 -0400

    fix: dont show error alert when user user mouse overs votes

    if they dont have permission to view votes

commit 24d0999fb5
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Thu Jun 6 13:49:14 2024 -0400

    fix(deps): update dependency pg-cursor to v2.11.0 (#12617)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit bee05fe212
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Thu Jun 6 13:28:59 2024 -0400

    fix(deps): update dependency pg to v8.12.0 (#12616)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-07 19:27:44 -04:00