The Southern Westerly Wind (SWW) belt is one of the most important atmospheric features of the Southern Hemisphere (SH). In Patagonia, these winds control the precipitation rates at the windward side of the southern Andes, and rainfall is very sensitive to any change (strength and/or latitudinal position) in the wind belt. The present-day behavior of the SWW, also known as westerlies, is characterized by remarkable seasonality. This wind belt also varies at interannual-to-decadal time scales, associated with the influence of atmospheric phenomena such as the El Niño–Southern Oscillation (ENSO) and the Pacific Decadal Oscillation (PDO), respectively. Moreover, during the past few decades, the westerlies have shown an increase in their core strength influenced by changes in the Southern Annular Mode (SAM). However, what controls the long-term variability of the SWW at the high latitudes of the SH is still a matter of debate. This work statistically analyzes the influence of large-scale modes of variability, such as ENSO and PDO on the SAM and the frequency of the strong SWW from ERA5 reanalysis data of southwestern Patagonia (~51°S), where the current core of this belt is located. Our results confirm the relation between strong wind anomalies and the SAM. In addition, the temporal variations of strong winds are also significantly affected by the PDO, but there is no detectable influence of the ENSO on their frequency. This shows that future studies focused on reconstructing wind history from aeolian particles of lake sediments from southwestern Patagonia could also provide information about the modes of variability that influence strong wind frequency.