Start Integration of Xcode Bot manually?

Question

I was watching WWDC 2014 \"Continuous Integration with Xcode\" Video and it looks great how the bots can be used to run test. But my question is to anyone who has seen the vide

Answer 1

Yes, as I answered here, you first need to find out the bot _id and then send a POST request to the bot's endpoint. See the link for details.

Answer 2

I want to add post-receive hook on github which on receiving any commit should start Xcode bot on my OS X Server.

If you want to 'build on commit' then just select that option when you create the bot. You have the option to run the bot Manually, Periodically or On Commit. The latter does what you describe. As soon as one of your team members commits a change to your github repo, Xcode server will do a build.

Answer 3

The prior two answers don't exactly answer the original question of "how they did that" to kick off bots from within Messages app.

I've recreated the exact workflow and scripts required to mimic the Jeeves virtual assistant for interacting with bots (and for getting the weather).

See the linked PDF document for the complete details:

https://s3.amazonaws.com/icefield/IntegratingXcodeBotsWithMessages.pdf

Edit: original answer was deleted due, I believe, to the fact that I referenced via a link to the full answer. This edit adds the full implementation details as part of this answer. I hope it is not too long for a SO answer.

Integrating Xcode Bots with Messages

During WWDC 2014 Session 415, Continuous Integration with Xcode 6, Apple demonstrated integrating Xcode bots with the Messages app via custom integration triggers. More specifically, starting at the 23 minute mark of that session’s video (https://developer.apple.com/videos/play/wwdc2014-415/), Apple demonstrates the use of integration triggers in conjunction with Messages to receive the status of integrations on a build server. Furthermore, through the use of a virtual chat room member, Jeeves, they demonstrate the ability to start integrations directly from within the Messages app. The following article provides step-by-step instructions to reproduce that functionality.

Client and Server Configurations

To get started, here are the configurations of the client and server I used to mimic the Jeeves functionality:

Client OS X Version 10.11 (El Capitan), Xcode 7.0.1

Server OS X Version 10.11 (El Capitan), OS X Server 5.0.4, Xcode 7.0.1, Ruby 2.0.0p645

Network For my development and continuous integrations, I use an internal network. My OS X Server is at domain.local, and my development machine is another node on the same internal network. The instructions below should work regardless if you’re using an internal or external server.

Jabber – The foundation of Messages

Jabber is the original name of an open source protocol for instance messaging. Jabber was renamed the Extensible Messaging and Presence Protocol (XMPP). The OS X Messages app is built using Jabber at its core.

We’ll use Jabber (Messages) extensively in this effort, so let’s ensure it is on. From the OS X Server App, select the Services > Messages view, and toggle on Messages in the upper right corner. For Jeeves, the Messages service settings I used are as follows:

From a terminal window on your server, if you want to check the specific settings for Jabber, use

$ sudo serveradmin settings jabber

Note in particular the jabberClientPortTLS (5222) and jabberClientPortSSL (5223) values. These are the ports on your server that you will use to communicate with the Jabber service.

We’ll be writing most of the scripts for Jeeves using Ruby, and we’ll need a XMPP/Jabber library to accomplish this. From a terminal window on your server, install XMPP4R (an XMPP/Jabber library for Ruby) using

$ gem install xmpp4r

Create Users for Jabber Service

Because my Server is a local server without any developer accounts on it, I needed to create accounts for various developers to login to Jabber. You may or may not need this step depending if your server already has user accounts defined.

From the OS X Server App on your server, go to the Accounts > Users list, and add new user for each client that will be utilizing the virtual Jeeves assistant. Be sure to create a new user for Jeeves. For user ‘Tom’, here are the settings that were used. Be sure to create an email address for each user, but the Mail service does not need to be running. These email addresses will be used to log into the Jabber service from the Messages App on your client.

Login to Jabber from Client Development Machine

With the user account(s) defined on your server, it’s now time to login to the Jabber account from your client machine. In the Messages app on your client, go to Messages > Preferences > Accounts. Select the + sign in the lower left, select “Other Messages Account...” and press Continue. In the Add a Messages account dialog, select Jabber for the Account Type, and fill in the credential information for your user(s). Here were the settings I used:

(Note with SSL toggled on, the Port (5223) matches the jabberClientPortSSL value you listed earlier when checking the settings of the Jabber service on your server.)

After successfully logging in to the Jabber service, you can optionally change your account nickname under the “Chat Settings” page of the Jabber account. All other default settings are okay to leave as is.

Create Chat Room

We want all bot integration statuses and communication to our virtual assistant, Jeeves, to be through a Messages chat room. Chat rooms allow group communication but you don’t need an invitation to join. To create the chat room, do the following.

From Messages, choose File > Go to Chat Room. You should see the account you logged into the Jabber service listed. Key in integration@rooms..local for the Room Name, and select Go. (Note that I found that chat room needed to be ‘rooms..local’ .com’>. Using a word other than ‘rooms’ would not create the chat room.)

Configure the Server Websites Service

When an integration is started from Xcode running on your client machine, the pre- and post- integration scripts communicate with the Jabber service by making an http call to a file on the OS X Server Website Service. You must configure the OS X Server Websites service to handle these calls.

You’ll need to modify the settings for the non-SSL http (port 80) site. Here are the settings I used.

Select the Port 80 Website, and select the pencil icon underneath to make your settings match these.

Select the “Edit Advanced Settings...” and make your settings match these. (Enabling “Allow CGI execution...” enables Ruby script execution.)

Finally, you’ll need to enable a particular file (message_room – we’ll discuss later) to be configured to run as a Ruby script. To do that, place the following .htaccess file in your web server’s default home folder (typically /Library/Server/Web/Data/Sites/Default).

Options +ExecCGI 
<FilesMatch message_room$>
    SetHandler cgi-script 
</FilesMatch>

NOTE: in all of the following ruby scripts, you’ll need to modify the variables just under the “credentials” comment in each script to match your domain, and login credentials.

Pre- and Post-Integration Scripts When we start an integration from Xcode on our client machine, we want to send a message to the Jabber Integration chat room so that all members of the chat room can be notified the integration has started (and finished). Add the following pre- and post-integration scripts to your project’s bot on the bot Triggers page within Xcode.

This is the pre-integration Trigger script:

#!/usr/bin/env ruby 
require 'json' 
require 'net/http' 
require 'uri'

# ------------------------------------------------------------------------------------- 
# credentials and such
domain = "<yourDomain>.local"

# ------------------------------------------------------------------------------------- 
# our messaging endpoint
uri = URI.parse("http://#{domain}:80/message_room")

# ------------------------------------------------------------------------------------- 
# what we want to say
message = "#{ENV['XCS_BOT_NAME']} integration #{ENV['XCS_INTEGRATION_NUMBER']} is now starting."

# ------------------------------------------------------------------------------------- 
# build up the request body
reqBody = {:message => message}
body = JSON.generate(reqBody)

# ------------------------------------------------------------------------------------- 
# the connect type
http = Net::HTTP.new(uri.host, uri.port)

# ------------------------------------------------------------------------------------- 
# build up the request
request = Net::HTTP::Post.new(uri.request_uri)
request.add_field('Content-type', 'application/json')
request.body = body

# ------------------------------------------------------------------------------------- 
# send the request and get the response
response = http.request(request)

This is the post-integration Trigger script:

#!/usr/bin/env ruby 
require 'json' 
require 'net/http' 
require 'uri'

# ------------------------------------------------------------------------------------- 
# credentials and such
domain = "<yourDomain>.local"

# ------------------------------------------------------------------------------------- 
# our messaging endpoint
uri = URI.parse("http://#{domain}:80/message_room")

# ------------------------------------------------------------------------------------- 
# what we want to say
integrationResult = case ENV['XCS_INTEGRATION_RESULT']
    when "succeeded"
        "has completed successfully."
    when "test-failures"
        tc = ENV['XCS_TEST_FAILURE_COUNT'].to_i
        "completed with #{tc} failing #{(tc ==1 ) ? 'test' : 'tests'}."
    when "build-errors"
        ec = ENV['XCS_ERROR_COUNT'].to_i
        "failed with #{ec} build #{(ec == 1) ? 'error' : 'errors'}."
    when "warnings"
        wc = ENV['XCS_WARNING_COUNT'].to_i
        "completed with #{wc} #{(wc == 1) ? 'warning' : 'warnings'}."
    when "analyzer-warnings"
        ic = ENV['XCS_ANALYZER_WARNING_COUNT'].to_i
        "completed with #{ic} static analysis #{(ic == 1) ? 'issue' : 'issues'}."
    when "trigger-error"
        "failed running trigger script."
    when "checkout-error"
        "failed to checkout from source control."
    else
        "failed with unexpected errors."
    end

message = "#{ENV['XCS_BOT_NAME']} integration #{ENV['XCS_INTEGRATION_NUMBER']} #{integrationResult}"

# ------------------------------------------------------------------------------------- 
# build up the request body
reqBody = {:message => message}
body = JSON.generate(reqBody)

# ------------------------------------------------------------------------------------- 
# the connect type
http = Net::HTTP.new(uri.host, uri.port)

# ------------------------------------------------------------------------------------- 
# build up the request
request = Net::HTTP::Post.new(uri.request_uri)
request.add_field('Content-type', 'application/json')
request.body = body

# -------------------------------------------------------------------------------------
# send the request and get the response
response = http.request(request)

The prior two Ruby scripts make a call to the message_room file residing in your OS X Server Website home folder (typically /Library/Server/Web/Data/Sites/Default). Place the following message_room file into that folder.

#!/usr/bin/env ruby
require 'cgi' 
require 'json' 
require 'xmpp4r' 
require 'xmpp4r/muc'

# ------------------------------------------------------------------------------------- 
# credentials and such
domain = "<domain>.local"
userId = "jeeves@#{domain}"
userPw = "<jeevesAccountPassword>"
roomName = "integration@rooms.#{domain}"

# ------------------------------------------------------------------------------------- 
# header sent back
cgi = CGI.new
puts cgi.header( "type" => "text/html", "status" => "OK")

# ------------------------------------------------------------------------------------- 
# get the message out of the json formatted text
keyValue = JSON.parse(cgi.params.keys.first)
key = "message"
value = keyValue[key] puts value

# ------------------------------------------------------------------------------------- 
# create the message to the iChat (jabber) room
fromJID = Jabber::JID.new(userId)
jabberClient = Jabber::Client.new(fromJID)
jabberClient.connect
jabberClient.auth(userPw)
jabberClient.send(Jabber::Presence.new.set_type(:available))

# ------------------------------------------------------------------------------------- 
# send the message to a chat room
roomID = roomName + "/" + jabberClient.jid.node
roomJID = Jabber::JID::new(roomID)
room = Jabber::MUC::MUCClient.new(jabberClient) room.join(roomJID)
roomMessage = Jabber::Message.new(roomJID, value) room.send(roomMessage)

Starting Integrations from the Messages App

We want to be able to issue instructions to our virtual assistant Jeeves from within the Messages App. We’re going to support three instructions:

1. Jeeves, weather # gets the current weather (w/o zip defaults to Cupertino)

2. Jeeves, integration (Bot Name) # starts an integration for the given Bot

3. Jeeves, exit # shutdown Jeeves on your OS X Server

The following files will be placed in your OS X Server website’s default folder (typically /Library/Server/Web/Data/Sites/Default).

The main file that handles the virtual assistant, Jeeves, is jeevesManager.rb. Start this file to wake Jeeves up by entering

$ ruby ./jeevesManager.rb

from the website’s default folder on your server.

#!/usr/bin/env ruby
require 'xmpp4r'
require 'xmpp4r/muc'
require 'xmpp4r/delay'
require './jeevesWeather.rb' 
require './jeevesIntegration.rb'

# ------------------------------------------------------------------------------------- 
# credentials and such
domain = "<domain>.local"
userId = "jeeves@#{domain}"
userPw = "<jeevesAccountPassword>"
roomName = "integration@rooms.#{domain}" 
defaultWeatherZipCode = "95015"

# ------------------------------------------------------------------------------------- 
# create the client we'll use
fromJID = Jabber::JID.new(userId)
jabberClient = Jabber::Client.new(fromJID)
jabberClient.connect
jabberClient.auth(userPw)
jabberClient.send(Jabber::Presence.new.set_type(:available))

# ------------------------------------------------------------------------------------- 
# connect to the chatroom
roomID = roomName + "/" + jabberClient.jid.node
roomJID = Jabber::JID::new(roomID)
room = Jabber::MUC::MUCClient.new(jabberClient) room.join(roomJID)

# ------------------------------------------------------------------------------------- 
# weather
def getWeather(m)
    begin
        words = m.body.downcase.split("weather") 
        where = defaultWeatherZipCode
        if (words.length == 2)
            where = words[1].strip 
        end
        weather = get_weather_for_city(where,'f') 
    rescue
        weather = "Couldn't get weather for that location - try zip code" 
    end
    return weather 
end

# ------------------------------------------------------------------------------------- 
# integration
def startIntegration(m)
    begin
        words = m.body.split("integrate") 
        botName = "Invalid BOT Name"
        if (words.length == 2)
            botName = words[1].strip 
        end
        integrationMessage = jeevesIntegration(botName) 
    rescue
        integrationMessage = "Failed integrating #{botName}" 
    end
    return integrationMessage 
end

# ------------------------------------------------------------------------------------- 
# listen for messages in chatroom (this callback will run in a separate thread) 
room.add_message_callback do |m|
    if (m.x.nil?) # the msg is current 
        if m.type != :error
            body = m.body;
            if (body.downcase.include? "jeeves")

                # assume Jeeves does not understand command
                understood = 0

                # exit Jeeves
                if (body.downcase.include? "exit") 
                    understood = 1
                    message = "Good-bye"
                    mainthread.wakeup
                end

                # Weather
                if (body.downcase.include? "weather") 
                    understood = 1
                    message = getWeather(m) 
                end

                # Integrate BOT
                if (body.downcase.include? "integrate") 
                    understood = 1
                    message = startIntegration(m) 
                end

                # Jeeves doesn't understand command
                if (understood == 0)
                    message = "I don't understand that command!"
                end

                # let user know what has happened
                roomMessage = Jabber::Message.new(roomJID, message)
                room.send(roomMessage)
            end
        end
    end
end


# ------------------------------------------------------------------------------------- 
# add the callback to respond to server ping (to keep the connect alive)
jabberClient.add_iq_callback do |iq_received|
    if iq_received.type == :get
        if iq_received.queryns.to_s != 'http://jabber.org/protocol/disco#info'
            iq = Jabber::Iq.new(:result, jabberClient.jid.node) 
            iq.id = iq_received.id
            iq.from = iq_received.to
            iq.to = iq_received.from
            jabberClient.send(iq) 
        end
    end 
end

# ------------------------------------------------------------------------------------- 
# stop the main thread (the call back will still be alive this way)
print "Connected to chat room...\n"
Thread.stop
print "Disconnected from chat room...\n"

# leave chat room and log out of Jabber
room.exit 
jabberClient.close

Two other supplemental files are utilized by the Jeeves manager file above. The first one below handles getting the weather forecast and formatting it, and the second handles starting an integration.

######### Weather #########
require 'rexml/document' 
require 'open-uri' 
require 'net/smtp'

# ------------------------------------------------------------------------------------- 
# yahoo weather url info
# http://developer.yahoo.net/weather/#examples

# ------------------------------------------------------------------------------------- 
#Returns a hash containing the location and temperature information
#Accepts US zip codes or Yahoo location id's
def yahoo_weather_query(loc_id, units)
    h = {}
    open("http://xml.weather.yahoo.com/forecastrss?p=#{loc_id}&u=#{units}") do |http|
    response = http.read
    doc = REXML::Document.new(response)
    root = doc.root
    channel = root.elements['channel']
    location = channel.elements['yweather:location']
    h[:city] = location.attributes["city"]
    h[:region] = location.attributes["region"]
    h[:country] = location.attributes["country"]
    h[:temp] = channel.elements["item"].elements["yweather:condition"].attributes["temp"]         
    h[:text] = channel.elements["item"].elements["yweather:condition"].attributes["text"] 
    h[:wind_speed] = channel.elements['yweather:wind'].attributes['speed']
    h[:humidity] = channel.elements['yweather:atmosphere'].attributes['humidity'] 
    h[:sunrise] = channel.elements['yweather:astronomy'].attributes['sunrise']
    h[:sunset] = channel.elements['yweather:astronomy'].attributes['sunset']
    h[:forecast_low] = channel.elements["item"].elements['yweather:forecast'].attributes['low']
    h[:forecast_high] = channel.elements["item"].elements['yweather:forecast'].attributes['high'] end
    return h
end

# -------------------------------------------------------------------------------------
def get_weather_for_city(city_code,units)
    weather_info = yahoo_weather_query(city_code, units)
    city = weather_info[:city]
    region = weather_info[:region]
    country = weather_info[:country]
    temp = weather_info[:temp]
    wind_speed = weather_info[:wind_speed]
    humidity = weather_info[:humidity]
    text = weather_info[:text]
    sunrise = weather_info[:sunrise]
    sunset = weather_info[:sunset]
    forecast_low = weather_info[:forecast_low] 
    forecast_high = weather_info[:forecast_high]

    return "#{city}, #{region}:\n" + " Currently #{temp} degrees, #{humidity}% humidity, #{wind_speed} mph winds, #{text}.\n" + " Forecast: #{forecast_low} low, #{forecast_high} high.\n" + " Sunrise: #{sunrise}, sunset: #{sunset}.\n"
end

Finally, this is the script that kicks off an integration from Messages app

require 'json' 
require 'open-uri' 
require 'openssl'

# -------------------------------------------------------------------------------------
def jeevesIntegration(botToIntegrate)

    # credentials
    domain = "<domain>.local"
    endpoint = "https://#{domain}:20343"
    user = "your-integration-username (not Jeeves)" 
    password = "password"

    # return message
    message = "Bot '#{botToIntegrate}' does not exist on server #{domain}"

    # request JSON construct with all the BOTS
    botsRequestURI = URI.parse("#{endpoint}/api/bots")
    output = open(botsRequestURI, {ssl_verify_mode: OpenSSL::SSL::VERIFY_NONE}) 
    bots = JSON.parse(output.readlines.join(""))

    # loop through full list of BOTS for the one we're interested in
    bots['results'].each do |bot| 
        botName = bot['name']
        if (botName.downcase == botToIntegrate.downcase) 
            botID = bot['_id']

            # curl -k -X POST -u "#{user}:#{password}" "#{endpoint}/api/bots/#{botid}/integrations" -i

            # ------------------------------------------------------------------- 
            # kickoff integration
            uri = URI.parse(endpoint)
            http = Net::HTTP.new(uri.host, uri.port)
            http.use_ssl = true
            http.verify_mode = OpenSSL::SSL::VERIFY_NONE
            request = Net::HTTP::Post.new("/api/bots/#{botID}/integrations")
            request.basic_auth(user, password)
            response = http.request(request)
            message = "Integrating #{botName} on server #{domain}" 
        end
    end

    return message 
end