You could keep a second, undisplayed image where you draw duplicates of the rectangles, each uniquely colored. When the user clicks on the picturebox, find the color of the corresponding pixel in the 2nd image, which will identify which rectangle was clicked.