The story
Sword is an old Germanic word. Old English sweord matches German Schwert, Dutch zwaard and Swedish svärd — and every one of those still pronounces the W-type sound after the S.
English did too, for a long time. The W was dropped in the same setting that swallowed it in two: before a rounded "o" vowel, where the lip rounding of the W and the vowel run together. The change was not sudden. In Early Modern English both pronunciations existed side by side — with W and without — and the W-less form eventually won in standard speech.
The spelling with w had been settled long before the pronunciation was. So sword now looks like swore and swarm but sounds like sored. Meanwhile the related words that did not have the rounded vowel next to the W — swear, sweep, swim — still pronounce it.
