The plugin provides 4, completely different, scripts – 2 of them being build with responsiveness in mind especially (SwipeBox and Nivo Lightbox). 2 other scripts are responsive too but are a bit older and may have not been tested for all, especially the latest, devices.
Which script you are using?