Building a University API: Why and How

UCL API is a student-led project working to expose the functionality of UCL services via an API that is easy to use for students and staff alike.

At present, it provides documented APIs for room bookings (with webhooks), the timetable, people search, library seat occupancy, desktop PC availability, and OAuth (Sign in with UCL).

If you’re a student at UCL and want to start building with these APIs, you can dive right in. To get a taste of some of the awesome tools, apps, and websites that folks have built with the API, take a look at the projects built at our first UCL API Hackathon. The Projects of UCL API Hackathon *Just about 4 weeks ago, we held our first UCL API Hackathon, and it was awesome! We had a lot of fun building apps on…*medium.com

This post has three parts, and we’d encourage you to read them one after the other, but feel free to skip straight to the final part if you’re in a rush.

Why build a university API?

For a few years now, most of the software that you interact with on a daily basis exposes some or all of its functionality via an API. No matter whether you’re writing emails in Gmail, listening to your favourite songs on Spotify, chatting to your co-workers in Slack or pushing code to GitHub, all of these services offer some form of API that you can use to connect them together or to have** **a computer** automatically perform some action** within each app.

As developers with too much free time and attending too many hackathons we were taking this very much for granted. “Want to play a song whenever you merge a pull request?” “Just use the Spotify and GitHub APIs!”. But as students at UCL, we regularly experienced a gap in the APIs that were available to us. Be it the virtual learning environment, the timetable app, or the map you’d use when trying to figure out where your lecture is —** none of these tools had APIs!**

The more UCL systems we used, the more frustrated we grew by a lack of APIs and therefore an inability to automate manual processes in a way that we had grown to expect. This inability became especially painful as part of running the UCL Technology Society, which frequently involved us looking for and booking rooms via a bizarrely inflexible, slow and generally pretty broken room booking system.

We started to wonder what a world would look like in which all UCL services came with an API. “Imagine if you could get a push notification right after a lecture finishes suggesting a number of available rooms that you can study in”, “Imagine being able to easily see which electives you can realistically select based on the timetable of your core modules”, “Imagine if you could automatically receive a notification in Slack when your room booking has been confirmed as opposed to getting an email 2 days later”.

Many of the things we imagined in late 2015 are now possible with the API we have built for UCL. The following section details our 2+ year journey (and counting!) of how a group of students collaborated with the UCL IT Department, built an API, and got paid for doing so.

Timeline

January 2016: First contact

Janina, a member of the UCL IT Department (known as the Information Services Division or ISD for short), reached out to the UCL Technology Society in hopes of establishing a better link between technical students and the IT department. A few of us from the TechSoc committee met to discuss how we could collaborate. We decided to run a meet up in the following month that would bring together both students and staff to kick off the conversation.

February 2016: First meet up

One of the many ISD meet upsOne of the many ISD meet ups

The first meet up, in what would become a more regular occurrence, was held in the UCL IT Department office. It was attended by both ISD staff and members of the UCL Technology society. For almost 2 hours we hammered staff with questions around how we could go about building an API and what it would take to expose data about the room bookings, timetable and Moodle (a virtual learning environment) services.

We were able to brainstorm with ISD staff and discovered a healthy appetite for student input on the things they were building. Many of them were extremely supportive right from the beginning and strongly sympathised with the idea of building an open API on top of UCL IT services.

The meet up ran again in March, June, and November 2016 as well as in February 2017. Throughout these meet ups we were able to build trust among many of the most relevant parties of the UCL IT department and started getting noticed by the people in charge.

April 2016: ISD All Staff Meeting

Pitch at the all staff meetingPitch at the all staff meeting

Once every few months the ~400 staff members of the IT department gather for a two hour “all hands” meeting where organisational changes are shared, important project updates are communicated, staff members give insight into their area of work, etc.

We were asked to present on the first student/staff meet up we had conducted in February and to extend the invitation to attend future meet ups to everyone in the room. We did talk about that, but we kind of hijacked the majority of our time slot to pitch the vision for UCL API.

The pitch was essentially the following:

If you’re curious, take a look at the slides we used for the pitch.

After the pitch I approached the director of ISD. He is in charge of the whole IT department, and we had had trouble getting a meeting with him directly. I asked him what he thought of the pitch and whether he wants to be our champion. He said “Yeah! Let’s make it happen! Email me to set up a meeting.”

Day. Made.

August 2016: The green light 💚

We kept talking, meeting, pitching, and discussing with senior staff in the weeks following the all staff meeting and things started to finally get serious in August of 2016.

For a while, we had been asking to get a single point of contact within the IT department who would be able to advocate for us internally, help us overcome administrative challenges, and point us in the right direction whenever we wanted to know or do something. The director himself was too busy to deal with us in a day to day capacity but one of the members of the ISD Architecture team stepped up to be “in charge” of us internally. He had come to some of our early meet ups and even though he didn’t like to show it, he was quite excited about what we wanted to do.

The ISD Architecture team would be instrumental to successfully building UCL API. This six person team is involved in almost every ISD project and as such has a great overview of everything that’s going on and who would be the right group of people to talk to no matter what we wanted to do. Being unofficially tied to this team was a massive advantage for us.

With our new single point of contact and a plan in mind we met with the director to discuss what we would need to get started in what would be the meeting that finally got us going.

We discussed what the first version of the API would look like, how it would work technically, how we as a team we’re planning to achieve that, and more. Throughout these discussions we made a strong case to use both the tools and the software stack that we were already familiar with, as opposed to what was in place by default at ISD. So Trello over JIRA, Python/Django/Nginx over PHP/Apache, AWS over ISD’s own infrastructure. As such we were lucky to steer clear of any slow provisioning/procurement processes, which helped us ship sooner and more frequently. We also made sure that whenever we needed a paid service (mostly AWS at the beginning), that ISD would be the one paying for it instead of us directly. Not just was this more financially sustainable but it also helped establish our legitimacy.

One of the things we also discussed at the August meeting was the potential of getting paid for our work. Right from the beginning the director was very open to this idea and encouraged us to push it forward. We decided that we didn’t want to get paid until we delivered our first API service (which happened in February of the following year). Not getting paid gave us the freedom to build this API in the way we wanted to build it as opposed to in the way that ISD might want us to. We were keen on proving our vision and technical ability before getting our first paycheque.

The team working at desks in the ISD officeThe team working at desks in the ISD office

Something else that we asked for in this conversation was access to the ISD office as well as desk space, so that we could work from there if we wanted to. Not just was it great to have a proper office to work from, but it also allowed us to build trust and have the serendipitous interactions with staff that ended up being so crucial in moving us forward.

At this stage everything was a go, we were a team of four people (all of us members of the Technology Society committee), but we still didn’t have access to the actual databases which contained the data that we wanted to make available via our API. What we did have was backing from the director, which was incredibly helpful in accelerating our request for access from the relevant teams.

December 2016: Roomie McRoomface launches (enghub.io)

While waiting for access to the data we needed from ISD we spent September to December 2016 trying our hands at a room booking system for the engineering faculty. This was a project that we could undertake with almost no external blockers and where we controlled the entire stack. It helped us learn about how to build software in an API-first way and was a great practice run before building the API we actually wanted to create.

Read more about Enghub

February 2017: UCL API Launch Party and Announcement of the Room Bookings API

Yes, of course we had branded cupcakes at our launch party!Yes, of course we had branded cupcakes at our launch party!

In the time from September 2016 to February 2017 we encountered a number of technical problems in building the API (mostly when our modern stack had to interact with existing enterprise systems). This was a nice change from the organisational and interpersonal problems that had been our primary challenge so far. Chris joined the team to help us with these problems and he’s blogged extensively about them: Adventures in Shibboleth and Nginx (Part 1 of 2) *How the Engineering Hub room booking system’s login system works, and how you, too, can build something cool with UCL…*medium.com Accessing Oracle Database via a Cisco VPN (via LDAP) on Linux *Two proprietary pieces of software in one title! Stallman would be triggered… (and yes, the solution is horrible)*medium.com

But in February 2017 we had overcome these problems and launched with our first API service: room bookings. This was a huge milestone for us! One year after we started talking about this we finally shipped an API that anyone in UCL could now build useful third party applications with.

The room bookings API returns data about which rooms are available and which bookings exist for which room by whom. It enables students to build things like a visualisation of which societies are booking the most rooms or a calendar overview of all your personal room bookings (shown below).

UCL Room bookings in Google Calendar (green)UCL Room bookings in Google Calendar (green)

These two applications of the room bookings API are good examples of how we built something to scratch our own itch. We wanted to automate away a part of the pain of managing a UCL society and finally we were able to!

View the UCL API Launch party slides

March 2017: We started getting paid

After delivering our first API we decided it was time to start getting paid. We started this conversation with ISD and faced zero resistance — after all, we had been working alongside them for several months now! Having delivered on what we promised we were also able to ask for a pretty decent hourly rate (around twice the rate of the typical student job in London).

October 2017: Launch of OAuth, Webhooks, Search API, and Timetable API

The summer months tended to be less busy as most of us were doing internships but in October 2017 several projects we had been working on for a while came to fruition.

Our OAuth offering makes it simple for any UCL website or app to offer a “Sign in with UCL” option. Your service can view data about a user’s faculty, department, and so on, all while the user experiences the Single Sign On flow that they’re used to. Neither UCL API nor your service needs to process or store any passwords.

We also introduced Webhooks for the room bookings API. After specifying which activity you want to be notified of, UCL API will automatically notify your service whenever there is a new room booking. Webhooks make it super simple to, for example, build an app that calls the TechSoc vice president whenever there is a new TechSoc room booking.

The Search API is a very simple API that returns a set of people at UCL given a query that you pass to it. As such it enables you to look up anyone you might interact with at UCL, for example as part of a Slack integration 👇

The Timetable API lets a UCL API app access student’s personal timetables (authorised via OAuth) or the timetable for a specific module. At the UCL API Hackathon, one app that used the timetable API calculated how late you can get up in the morning and still make it to your first lecture in time. Another helped you with selecting electives based on whether they clash with your core modules.

October 2017: UCL API Hackathon

As already mentioned a few times, in October 2017 we also had our inaugural hackathon focused on UCL API. We had a ton of fun over the two days and marvelled at the creativity of the projects that came out of it.

November 2017: Launch of Resources API

The Resources API exposes the availability of desktop machines in various UCL computer labs and was added to our growing list of APIs in November 2017.

March 2018: Launch of Workspaces API

Screenshot of ‘UCL Assistant’ using the Workspaces APIScreenshot of ‘UCL Assistant’ using the Workspaces API

The Workspaces API shows granular availability of individual seats across UCL libraries. While not incorporated into many projects yet, this API will allow apps to show detailed information and make informed recommendations about the ideal library or study space that students should go to at any given time.

5 Tips if you want to build an API at your university

Building a university API has been a fun challenge for us so far, but we’re far from done! The following section includes a few pieces of advice that we have gathered over the last 2+ years. Hopefully these tips can help you in building an API for your university.

Build trust

Building trust is by far the most important piece of advice we can offer. You’re a group of students trying to convince staff of something that is risky for them to approve without any tangible benefit AND you’re also asking for money to do it.

In our experience, the only way to overcome this perception of high risk and lack of tangible benefit is time. As you spend time with staff, listen to them, continuously repeat what you’re trying to accomplish, and view the situation from their perspective, many of these seemingly insurmountable problems will go away.

Fundamentally what you are doing is building trust with staff. Trust in your ability to actually pull this off technically. Trust in your ability to not create a huge security vulnerability. Trust in that this project won’t just be dumped when you graduate. At some point, staff may even realise that this student-led, outside-the-box project *could *make them look good in the eyes of their superiors. That’s the point you want to get to!

The only problem with this piece of advice is that it requires time, and lots of it! From our initial interaction with the IT department it took 3 months to get in front of the right people, 7 months to get taken seriously, 12 months before we managed to deliver on the first part of our original vision, and another 4 after that to get paid.

Try to ignore the doubters

People will keep telling you that what you’re doing won’t happen, because how could it?!?!? Some folks gave us long lectures about problems that we hadn’t yet encountered but that we absolutely will and that would slow us down tremendously or even render the whole effort a failure. We never encountered those problems.

It’s quite normal for people to doubt you and to fear the change that you are planning to enact. As long as not literally everyone is doubting you (and chances are they’re not) un-constructive advice is safe to ignore.

Have a single point of contact

Having a single point person in a department that, as students, we were naturally somewhat external to was incredibly important and evolved very much into a collegial relationship. Whenever we had a question about something, we could ask our colleague. When there was someone we wanted to meet, we could rely on our colleague to broker that introduction. Not having a single person “on the inside” would have meant much more poking around in the dark for us, which would have slowed us down tremendously.

Nosh, if you’re reading this, thank you so much! You were awesome!

Find out who your allies are

Throughout our many meetings, conversations, pitches, and occasional drinks keeping track of who liked working with us and who supported what we were doing was useful whenever we wanted to get anything done.

Being a somewhat unconventional team in what is essentially a very conventional, process heavy university, our allies were how we got almost anything done. This could be in the form of access to certain systems, configuration changes, small modifications to existing systems that we needed, various logistical acrobatics that any hackathon requires, etc. etc.

We rarely went the official way, which required project proposals, detailed budget plans, and lengthy decision processes. This was awesome and reduced the time we needed to spend to a manageable amount — after all we were also all enrolled (or still are) in full time study while building this API. Building UCL API absolutely would not have been possible without our allies.

Don’t get institutionalised

We always believed that the perspective we could provide, as students, on the problems that the IT department wanted to solve was one of our biggest advantages. “Who better to improve the student experience than students themselves?” is something that we argued a lot.

One line of thinking that didn’t occur to us for quite a while is that, yes, we were students, but we were also now part of an institution and with it acknowledged and adapted (some of) its rules, processes, procedures, and ways of working. Even if you try very hard to stay independent, no team or individual can sustainably be productive in an environment that they are at odds with. You adapt and become institutionalised — it’s normal.

A good antidote to this institutionalisation is bringing on new people to your team. Folks who still have that fresh, outside perspective and are willing to say: “No, it’s not normal to have to put in a request and wait two weeks to have cURL installed on your server” or “What do you mean it takes 6 months before we can see the results of the survey we did last week? That’s ridiculous!”.

TL;DR: build trust, try to ignore the doubters, have a single point of contact, find out who your allies are, and don’t get institutionalised.

Thank you

Thank you to everyone who helped us come this far! First and foremost the entire UCL API team, the ISD architects, and all the allies in ISD who helped us in so many ways over the years. Especially all of the enthusiastic early supporters! Thank you also to everyone in TechSoc who put in so many hours to support UCL API, and thank you finally to all the friends who offered advice, counsel, and help in many shapes and forms on this wonderful journey.

The future

In January 2018 I handed over the leadership of UCL API to Chris in preparation for my graduation. Read more about his plans for the future of UCL API in an upcoming blog post on this publication ✨

UCL API is also hiring! If you’re a first or second year student at UCL and interested in joining the team either as a software engineer or in a marketing role, we’d love to hear from you!

This post was originally published on medium.com

#UCL API