There certainly seems to have been a garrison. One web page has a couple books about the period even. Even seem to have built 3 forts.
I have not received a response from the tourism office but there does appear to be a group called St. Barths Association of the Friends of Sweden (ASBAS)
To Join the Company of Military Historians click here