Skip to content

NodeBB Global Tchat : Forum with many users performance issues

Solved Configure
  • @phenomlab said in NodeBB Global Tchat : Forum with many users performance issues:

    I’d say that this was still relevant, and would solve the short term issue (in most cases) but if anything, you’ll still experience slowness if you have say 600 users online.
    The right way to handle this in my view would be to pass the notification stream to a message queuing service (like RabbitMQ or Redis) and let that process in the back end rather than cause locking on the forum itself. I’m surprised that this is in fact the case as standard emails are being queued and aren’t sent in real time.
    I personally don’t use this plugin as you know, so I’d need to review the code first in order to understand it’s structure.

    Thank you for your analyse 👍

    I would have liked to try the thing all the same, just to see the result

    600 users is better than 4000 😉

    Here is the content of the /src/messsagin/notification.js file

    I put it here because I don’t see where to integrate the given code for test :

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    };
    
  • @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

  • @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

  • @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

  • @DownPW you can always experiment 👍

  • @DownPW what have you tried?

  • @phenomlab lot of things 😉

  • @DownPW can you provide some brief examples?

  • @phenomlab .

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    		
               if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
                   Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
                } else {
                    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
                }
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    

    nodebb build is ok with this code but I don’t see any diiference of latencies

  • 'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    		
    		if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
            Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
            } else {
            user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
            }
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    
  • DownPWundefined DownPW has marked this topic as solved on

Did this solution help you?
Did you find the suggested solution useful? Why not buy me a coffee? It's a nice gesture, and a great way to show your appreciation 💗

  • Composer options on nodebb

    Solved Configure
    8
    3 Votes
    8 Posts
    287 Views

    @Panda You should be able to expose the CSS for these using F12 to get into console

    3591518c-e3a3-4ada-a43c-6b32a5e0359c-image.png

    a2b8ed46-4157-4ff2-85f0-576543380107-image.png

    That should then expose the element once selected

    89d9c545-a47a-40d1-98f4-80cf3b958e8f-image.png

    Here’s the below CSS you need based on the screenshot provided.

    .composer .formatting-bar .formatting-group li[data-format="picture-o"], .composer .formatting-bar .formatting-group li[data-format="spoiler"] { display: none; }
  • NodeBB v3 Android Problem

    Solved Configure
    4
    4 Votes
    4 Posts
    189 Views

    thank you fixed.

  • NodeBB: Favicon upload issue

    Solved Configure
    12
    3 Votes
    12 Posts
    356 Views

    @phenomlab I am on a Mac, so I used the “Option + Command + I”, and then performed the steps. It loaded my favicon! I checked on Firefox which I haven’t used before, and it showed my favicon also! That’s fantastic and thank you for the help!

  • 0 Votes
    5 Posts
    193 Views

    @mventures Yes, exactly. The other icon will restart NodeBB whilst the first icon I referenced will rebuild (recompile) it.

    The huge strength of NodeBB over Flarum (for example) is that the code is precompiled, and called once at boot. PHP’s code has to repeatedly reload code from source making it much slower.

  • Issues with v3 alpha and Harmony

    Moved Solved Configure
    18
    4 Votes
    18 Posts
    414 Views

    @cagatay Should be all sorted now 🙂
    Don’t forget to put your language back to Turkish.

  • 1 Votes
    2 Posts
    158 Views

    @phenomlab

    In fact I specified the sub rep and not the rep

    DON’T DO THIS:

    nodebb@nodebbpwclonedb:~/nodebb$ sudo mongorestore --username admin --password XXXXXXXXXXXXXX --nsInclude nodebb.objects --drop /home/nodebb/nodebb_DB_20230107/nodebb/

    BUT THIS :

    nodebb@nodebbpwclonedb:~/nodebb$ sudo mongorestore --username admin --password XXXXXXXXXXXXXX --nsInclude nodebb.objects --drop /home/nodebb/nodebb_DB_20230107/

    🙂

  • Podcast Share NodeBB

    Solved Configure
    15
    4 Votes
    15 Posts
    483 Views

    @cagatay You could experiment with nodebb-plugin-ns-embed but I expect the x-origin tag on the remote site to prevent playback.

  • nodebb dropdown menu

    Solved Configure
    5
    0 Votes
    5 Posts
    428 Views

    @phenomlab said in nodebb dropdown menu:

    @kurulumu-net You set it like the below example taken from this site

    aae36790-3257-4bb2-ad5a-0d744309876a-image.png

    Which presents this

    77f47260-2941-4afe-9614-8e17dcfc8c19-image.png

    Very interesting…

    I actually thought this wasn’t possible, as I remember it being asked in the NodeBB forum.

    Is this something new that’s been implemented? I’ll 100% be doing that when I’m on the laptop over the weekend.