Commit Graph

530 Commits

Author SHA1 Message Date
Barış Soner Uşaklı
75816deca7 add cache to redis (#6917)
* add cache to redis

move out cache module from mongo

* fix redis tests

* add callback noop

* fix typo

* del cache on field delete

* make redis/mongo caches separate
2018-11-07 15:04:56 -05:00
Barış Soner Uşaklı
0de60cfdca fix mongodb tests 2018-10-30 18:58:02 -04:00
Barış Soner Uşaklı
8ca36ad122 closes #6890 2018-10-30 18:29:52 -04:00
Barış Soner Uşaklı
7aa937a657 remove dupe code in mongodb sorted 2018-10-28 16:20:38 -04:00
Barış Soner Uşaklı
c27be9db5a Recent refactor (#6879)
* wip

* fix inf scroll

* remove duplicated code

* remove dupe code in /unread

* use topicList

* update tag page to use topicList

* fix tests

* combine ifs

* remove more dupe code

* disable timeout
2018-10-27 06:26:50 -04:00
Baris Usakli
51792da430 dont crash if there is nothing to write 2018-10-26 15:15:20 -04:00
Barış Soner Uşaklı
fb6067a7ca only parse if field is requested 2018-10-25 19:58:01 -04:00
Baris Usakli
138154a71c move modules 2018-10-23 15:03:32 -04:00
Barış Soner Uşaklı
b2b33ffa57 topics data refactor 2018-10-20 16:10:02 -04:00
Barış Soner Uşaklı
26d4e0852f use includes instead of indexOf
use _.uniq instead of filter&indexOf
2018-10-20 14:40:48 -04:00
Barış Soner Uşaklı
986511e718 try with higher timeout 2018-10-16 21:43:40 -04:00
Barış Soner Uşaklı
1d15dc089b revert isCluster 2018-10-16 20:34:49 -04:00
Barış Soner Uşaklı
0aebb9caf8 closes #6844 2018-10-16 20:27:14 -04:00
Barış Soner Uşaklı
154d7076d9 dont pass options 2018-10-16 19:57:24 -04:00
Barış Soner Uşaklı
9dbe68a9cb testing again 2018-10-16 19:46:55 -04:00
Barış Soner Uşaklı
aa567a5f2d print out connString and opts 2018-10-16 19:12:33 -04:00
Barış Soner Uşaklı
2d8d234c57 test mongodb again 2018-10-16 18:44:03 -04:00
Barış Soner Uşaklı
e615391b22 convert values to string before removing 2018-10-11 16:23:05 -04:00
Barış Soner Uşaklı
18d0deeccb update deprecated methods (#6794) 2018-09-26 15:02:57 -04:00
Ben Lubar
11b655163e Only run the DDL for session storage on the primary instance. (#6788)
This fixes a restart loop caused by deadlocks in large clusters.
2018-09-25 12:12:18 -04:00
Barış Soner Uşaklı
9724ef083e remove v from query 2018-09-24 16:59:12 -04:00
Barış Soner Uşaklı
786311fa64 use value 2018-09-24 16:58:35 -04:00
Barış Soner Uşaklı
5a69f979a8 update processSortedSet to use withScores 2018-09-24 16:57:19 -04:00
Ben Lubar
97a3079bc0 Optimize PostgreSQL session store.
- Sets session ID to be a constant-sized character field inline in the
  table instead of a variable-length character field compressed outside
  of the row.
- Changes the session data from "json" to "jsonb" which is faster and
  smaller on disk with the drawback of not maintaining key order and
  whitespace (which we don't care about at all)
- Adds a clustered index on the session expiration timestamp, which will
  make periodic session purging much faster.

(For an example, WTDWTF currently has 482355 sessions in the table, and
 because there is no index on the expiration timestamp, PostgreSQL must
 check all the rows in the table every time.)

This upgrade script can run concurrently with a live NodeBB instance,
but any action that touches sessions will wait until the upgrade script
is done.

Does not touch the database if the database driver is not set to
PostgreSQL or if there is a Redis database configured.
2018-09-02 00:32:55 -04:00
Barış Soner Uşaklı
0519f84734 Promisify modules (#6723)
* WIP promisify

* promisify psql

* ability to skip some keys

* dont promisify client object

* remove async

* clone entire module so it has all properties

* add shim for node 6

* ignore sessionStore as well

* ignore pool on psql
2018-08-31 11:04:42 -04:00
Ben Lubar
33228bb7fe PostgreSQL database driver (#5861)
* [test/database/list] Fix test list 4 being used in two different tests

* [database/postgres] PostgreSQL database driver

* [database/postgres] Make transactions work based on continuation scope.

* [database/postgres] Implement nested transactions

* eslint --fix

* Add database changes from earlier this week to the PostgreSQL driver.

* Fix typo

* Fix postgres.incrObjectFieldBy returning undefined instead of null when given NaN

* [database/postgres] Fix sortedSetsCard returning an array of strings.

* Update socket.io postgres adapter

* Fix PostgreSQL erroring when multiple updates are made to the same sorted set entry in a single operation.

Add a test case to catch this error.

* Fix lint errors.

* Only prune sessions on one instance in a cluster to avoid deadlocks.

They're caught and handled by the database server, but they spam the logs.

* Fix arguments.slice.
2018-08-08 15:13:48 -04:00
Baris Usakli
16a7728ab8 fix redis warnings 2018-07-06 14:20:37 -04:00
Barış Soner Uşaklı
fe80186ddf add withscores 2018-06-22 16:28:22 -04:00
Barış Soner Uşaklı
8263f81d0c fix pubsub when only mongodb is used 2018-05-25 11:56:40 -04:00
Barış Soner Uşaklı
3b5d6951bb closes #6499 2018-05-10 16:31:16 -04:00
Barış Soner Uşaklı
77d47b31fb cleanly shutdown
wait for webserver to stop accepting connections
destroy current connections
wait for db connection to close
2018-03-22 16:36:23 -04:00
Baris Usakli
3c4b4f5263 don't call calback twice #6375 2018-03-21 12:25:25 -04:00
Barış Soner Uşaklı
be5c237388 fix decr/incrObjectField if key in array doesn't exist create it 2018-03-17 19:08:52 -04:00
Barış Soner Uşaklı
6a9a73c86c changes to leaveAllGroups
Groups.destroy can take an array of groupnames
Groups.leave can take an array of groupnames
db.incrObjectField/decrObjectField can take an array of keys
db.sortedSetRemove can take an array of keys and values
db.setRemove can take an array of keys
2018-03-17 18:49:38 -04:00
Baris Usakli
38acedf5bb lint 2018-03-15 12:18:54 -04:00
Baris Usakli
959a742c63 add llen to dbal 2018-03-15 12:06:40 -04:00
Barış Soner Uşaklı
bce640e514 closes #6340 2018-03-03 12:31:27 -05:00
Barış Soner Uşaklı
7a2d6fe22d Merge remote-tracking branch 'refs/remotes/origin/master' into develop 2018-02-02 10:36:25 -05:00
Barış Soner Uşaklı
b084a596fa closes #6301 2018-02-01 14:33:06 -05:00
Barış Soner Uşaklı
c0b7d4f5b5 fix redis export 2018-01-22 16:51:12 -05:00
Barış Soner Uşaklı
8a6345ded7 possible fix for pubsub 2018-01-22 16:41:22 -05:00
Ben Lubar
e85aabbe74 Allow running as a cluster without Redis (#6233)
* [database/*] Allow databases other than Redis to provide pubsub for clustering if Redis is not present

* [pubsub] Delay messages sent before the database is ready until the database is ready.

* [pubsub] Restore old behavior of not using the database in non-clustered NodeBB instances.

See comment: https://github.com/NodeBB/NodeBB/pull/6233#issuecomment-357814968
2018-01-18 13:02:56 -05:00
Julian Lam
457194b333 Merge branch 'master' into develop 2018-01-17 11:02:02 -05:00
Barış Soner Uşaklı
e092778ba6 closes #6200 2018-01-15 15:05:33 -05:00
Ben Lubar
5b1ed21634 [database/mongo] Improve speed of sortedSetRank (#6229)
* [database/mongo] Improve speed of sortedSetRank

* [database/mongo] Fix sortedSetRank to filter by _key
2018-01-12 13:32:07 -05:00
Barış Soner Uşaklı
5146f43b33 remove console.log 2018-01-03 14:38:09 -05:00
Barış Soner Uşaklı
87d40e9e78 fix increment to use data 2018-01-03 14:24:07 -05:00
Barış Soner Uşaklı
cbaa977233 fix type on mongodb if key has expireAt 2017-12-30 11:41:36 -05:00
Barış Soner Uşaklı
ea8cf6545c change db.set/get to use data field instead of value 2017-12-18 20:21:38 -05:00
Barış Soner Uşaklı
8e04e86e24 closes #5912 2017-12-07 20:43:07 -05:00