Hillsborough does a good job explaining what happened at Tenmaya, but surprisingly he left out one very important detail. He never explained how it was that the Tosa ronin managed to get into the shop in the first place.
The answer is that the Shinsengumi did not know that the people they were opening the door to were from Tosa.
The Shinsengumi’s task was to protect an important clansmen of the Kii domain, Miura Kyutaro, whom the Tosa men assumed had been behind Sakamoto’s death. Their belief was due to an incident where a Kii vessel had collided with one of Sakamoto’s ships and the domain had been obliged to pay out a very large sum of money. It was assumed that Miura had arranged for the Shinsengumi to kill Sakamoto in retribution for this incident. (They were wrong of course. Both Miura and the Shinsengumi were innocent.) Learning that his life was in danger, Miura had arranged through Aizu for the Shinsengumi to protect him and they hid out at the Tenmaya inn.
Had anyone with a Tosa accident come to the door of the inn, the Shinsengumi would have been instantly on their guard. However the person who eventually identified himself spoke with another accent altogether.
Mutsu Munemitsu was not from Tosa like the rest of the attacking party, but he had been a good friend of Sakamoto’s and an important member of his Kaientai. And as it just so happened, Mutsu was originally from the Kii domain. (He had abandoned the clan and joined the loyalists cause due to an injustice done to his father.) When the group that was seeking revenge on Miura arrived at Tenmaya, it was Mutsu who announced them. Because the Shinsengumi heard a Kii accent, they assumed it was someone they could trust and opened the door for their enemies!