I work for a school that has an institutional YouTube account (Google Apps for Education). The video privacy options are Public, Unlisted, and Private
After reviewing the YouTube Data API v3 documentation, I have come to the conclusion that this is not possible with the current API.
This may be related to Issue 6265 that is currently reported as an API defect. If this does not address your use case I strongly suggest you file an enhancement ticket to gdata-issues.