I don’t think you can do it with a shortcode button, unless it has an option to add rel attribute to an image link output.
Basically, if you are using WP gallery or insert image link into post, Responsive Lightbox should handle that automatically. Otherwise you need to add rel=”lightbox” to image links in yout theme.