Your <a> reference supplies target="_blank" which will cause the new window.
Not being sure how you are encoding the link I will provide what works for me:
Code:
<div><a rel="lightbox[product-PROD_RN]" style="display: none" title="Your Title" href="http://www.manzanitalab.com/sc_images/products/1024_scaled.jpg">1</a>
<a rel="lightbox[product-PROD_RN]" style="display: none" title="Your Title" href="http://www.manzanitalab.com/sc_images/products/1024_scaled.jpg">2</a></div>
This is the code I use to add TWO additional pictures to the Large Image which is displayed by default. To add just one pic, delete appropriately. Too add more, just expand the div.
I just add to the end of the Product description content.
Hope this helps.
Phil
Tucson