The question I want to raise is whether branching and merging are good tools for an agile development team, or a nuisance.

IMHO, the short answer is nuisance, which is why I push to merge back into the main line at least once a day and try to push/pull just as often.

Branches are inherently about creating isolation

I also agree with this statement which is why feature toggles are an excellent way to create new features without allowing them general adoption.