apache kafka – When we want to scale messaging systems horizontally how is it normally done?

I am reading about messaging systems and message brokers and I am confused on the following:
Sometimes when I read the various sources they seem to use the term “queue” and “broker” interchangeably and others times not.
So for instance it is not clear to me if we pick a specific message broker (e.g. ActiveMQ or Kafka or whatever) if we want to horizontally scale it: does that mean we need to:
a) run more broker instances? or
b) configure more queues while still having one broker instance?

If to scale we need to do (a) when exactly is (b) needed and when is it done out of the box by the broker vs the application or the admin to setup more queues for one broker?