Skip to content

NodeBB Global Tchat : Forum with many users performance issues

Solved Configure
  • Like you know I use glolbal chat plugin
    We had a massive influx of registrations and new members which means that they all integrate the chat room of the plugin

    We see this topic on github plugin page :

    https://github.com/NodeBB/nodebb-plugin-global-chat/issues/5

    I see maybe a solution here dating from 2019 : https://github.com/NodeBB/nodebb-plugin-global-chat/issues/5#issuecomment-492242438

    7bc1e6f9-542e-41d9-96aa-8a717d6bd538-image.png

    I then asked the question to the staff without an answer :

    Is this still relevant?
    Where in the file should I put the code?

    So I’m opening this topic to find out what you think about it with your expert eye 😉

  • @DownPW I’d say that this was still relevant, and would solve the short term issue (in most cases) but if anything, you’ll still experience slowness if you have say 600 users online.

    The right way to handle this in my view would be to pass the notification stream to a message queuing service (like RabbitMQ or Redis) and let that process in the back end rather than cause locking on the forum itself. I’m surprised that this is in fact the case as standard emails are being queued and aren’t sent in real time.

    I personally don’t use this plugin as you know, so I’d need to review the code first in order to understand it’s structure.

  • @phenomlab said in NodeBB Global Tchat : Forum with many users performance issues:

    I’d say that this was still relevant, and would solve the short term issue (in most cases) but if anything, you’ll still experience slowness if you have say 600 users online.
    The right way to handle this in my view would be to pass the notification stream to a message queuing service (like RabbitMQ or Redis) and let that process in the back end rather than cause locking on the forum itself. I’m surprised that this is in fact the case as standard emails are being queued and aren’t sent in real time.
    I personally don’t use this plugin as you know, so I’d need to review the code first in order to understand it’s structure.

    Thank you for your analyse 👍

    I would have liked to try the thing all the same, just to see the result

    600 users is better than 4000 😉

    Here is the content of the /src/messsagin/notification.js file

    I put it here because I don’t see where to integrate the given code for test :

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    };
    
  • @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

  • @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

  • @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

  • @DownPW you can always experiment 👍

  • @DownPW what have you tried?

  • @phenomlab lot of things 😉

  • @DownPW can you provide some brief examples?

  • @phenomlab .

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    		
               if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
                   Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
                } else {
                    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
                }
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    

    nodebb build is ok with this code but I don’t see any diiference of latencies

  • 'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    		
    		if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
            Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
            } else {
            user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
            }
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    
  • DownPWundefined DownPW has marked this topic as solved on

Did this solution help you?
Did you find the suggested solution useful? Why not buy me a coffee? It's a nice gesture, and a great way to show your appreciation💗

  • Fixed background to nodebb forum

    Solved Configure
    25
    4 Votes
    25 Posts
    1k Views

    @Panda said in Fixed background to nodebb forum:

    Chatgpt told me the ::before method.

    Go figure 😛

  • NodeBB v3 Quick reply

    Solved Configure
    4
    2 Votes
    4 Posts
    168 Views

    here is the link: https://app.transifex.com/nodebb/nodebb/translate/#tr

  • 0 Votes
    2 Posts
    120 Views

    @mventures Quick reply is basically exactly that. However, it’s possible to trigger the full composer as below

    82ca1209-31be-4a51-a641-9887b5a238b8-image.png

    Failing that, you’ll need to use the reply button in the sticky toolbar

    1d230bcb-ff8e-4756-94f7-6e2fb7a94bc4-image.png

  • 0 Votes
    6 Posts
    214 Views

    @mventures You’d need to connect to the server and execute it directly - not on your local terminal. Review the guide below, which will show you how to gain access via SSH to your server

    https://docs.ovh.com/gb/en/dedicated/ssh-introduction/

    Once you have access, you’ll need to navigate to the actual folder where NodeBB is installed

    You’ll then need to change to the directory as shown below

    /home/unbuntu/nodebb

    fdffe673-bf63-4b6d-a728-5506fddc1aff-image.png

    In most cases, initial access takes you to the root of the file system. You can always issue pwd in a Linux terminal which will show you the Present Working Directory. From there, you can issue the command

    cd /home/ubuntu/nodebb

    Once in the NodeBB directory, you’d use the below commands

    ./nodebb stop git fetch && git checkout develop && git reset --hard origin/develop ./nodebb upgrade ./nodebb start

    Line 1 stops the NodeBB instance
    Line 2 gets the latest files from GIT (repository) and then checks out the development branch. It then resets the version you are using to the development branch ready for v3
    Line 3 Runs the upgrade once the new branch is set, and code pulled
    Line 4 Restarts the NodeBB instance after the upgrade has completed

    Note that when you restart NodeBB and log back in, things will look very different to what you had in v2.

  • Issues with v3 alpha and Harmony

    Moved Solved Configure
    18
    4 Votes
    18 Posts
    400 Views

    @cagatay Should be all sorted now 🙂
    Don’t forget to put your language back to Turkish.

  • 2 Votes
    19 Posts
    556 Views

    @phenomlab Work now 😉

  • restarting nodebb on boot

    Unsolved Configure
    3
    1 Votes
    3 Posts
    214 Views

    @eeeee said in restarting nodebb on boot:

    can I just run nodebb under nodemon for auto restarts?

    It’s a better method. Nodemon just looks for file system changes and would effectively die if the server was rebooted meaning you’d have to start it again anyway. Systemd is the defacto standard which is how the operating system interacts in terms of services, scheduled tasks etc.

  • 3 Votes
    39 Posts
    1k Views

    @DownPW said in Virtualmin SQL problem:

    Finally problem solved simply but I still don’t understand why this service was installed.

    Glad to hear (and see) that this issue is now resolved. Virtualmin and Webmin are both derived from the origin of LAMP (Linux, Apache, MySQL, and PHP) - the earliest form would have been a project which was a fork of the original concept called “WAMP” (Windows, Apache, MySQL, and PHP)

    https://www.wampserver.com/en/

    Scroll to the bottom, and you’ll see the packages it comes with

    cf3c0965-a699-4c6f-b89f-65e7bb381bbc-image.png

    Over time, activity on this project dropped somewhat due to the rise of Virtualmin and Webmin - acting as “full blown” platforms designed to manage an entire web server from start to finish, and providing an easy way to do so with a GUI interface. Over time, the LAMP bundle became LEMP (Linux, NGINX [actually pronounced “engine X”], MySQL, and PHP). The issue with Apache is that it had limits, and compared to NGINX, was in fact much slower.

    Virtualmin and Webmin do not actually use MySQL for their core operation - they don’t even use Apache or NGINX for the core web services, which is why the admin port is set to a default of 10000 when you first complete the setup.

    Essentially, you can think of Virtualmin and Webmin as a central pane of glass to manage the underlying components that are required to run a website. MySQL doesn’t need to be installed if you are not actually using it, but is there as PHP typically is paired with either MySQL or MariaDB, so it makes sense to offer the installation of this itself, as well as support for managing it.