Decrypting JSON Web Encrypted tokens in PCL

This week I worked on a feature for a Xamarin app I’m building that gave me quite some headaches because it involved a good amount of yak shaving to get it working correctly. My main reason for blogging this is for my own reference in the future, but maybe it can help someone else too ūüôā

yak shaving
Yak shaving, gotta love it…

Our team is building a cross platform Xamarin app that has a good amount of shared code in an MVVM architecture. Most of the app is implemented in PCL libraries with a front end project per platform.

For communicating with a backend API, the app requires a token obtained from an Identity Provider. In addition to the API access token, we’re also getting an OpenId token that is crucial for the app.¬†Using Xamarin.Auth for example, it’s pretty easy to setup an OAuth flow that lets the user login and the app obtain that token.

So far so good.

There’s a lot to be said about which OAuth flow is suitable in which scenario. For us, embedding a client secret in the app was a no go, since this can’t be kept secret if it’s embedded in the source code of the app (reverse engineering!). We chose a PCKE approach. Without going into too much detail around the flow, I want to focus on one aspect: securing¬†the OpenID token. This blogpost is mostly about what I learned about token encryption and decryption.

207h

Our requirement was to have asymmetric encryption in place, where the app provides a public key to the server, which it uses to encrypt the JWT. The app can then decrypt the data using its private key, so we could mitigate a token interception attack. The public key was to be transferred in an X509 certificate (PEM encoded).¬†The idea was to have the app generate a private/public keypair on-the-fly as we don’t want to hard code a private key into the app. That would lead us to the same situation as the hard coded Client Secret. All we want is to make sure that the requesting app is the only one able to decrypt the token it requested.

There’s a nice standard for encrypting tokens using a variety of algorithms:¬†JWE – JSON Web Encryption. The original token is in a JSON structure, encrypted¬†and packaged into a standardized format. In our case, these were the specs:

Algorithm: RSA_OAEP РRSAES using Optimal Asymmetric Encryption Padding (OAEP) (RFC 3447), with the default parameters specified by RFC 3447 in section A.2.1

Encryption method:  A256GCM РAES in Galois/Counter Mode (GCM) (NIST.800-38D) using a 256-bit key

Basically what happens is: the token payload is encrypted using a random Content Encryption Key (CEK) provided by the server. This is an authenticated encryption mode, where data is added to prove authenticity. In addition to the key, an Initialization Vector (IV) is added, and also an Authentication Tag (authTag) and Additional Authenticated Data, basically consisting of a Base64 representation of the JWE header. All this together provides a symmetric encryption through which we can decrypt the token. This is the A256CGM – AES step.

All the information I mentioned – the CEK, IV, cipherText (the encrypted token), AAD and authTag – are present in the JWE package. To add the final layer of security, the CEK has to be encrypted, otherwise anyone would be able to read the token. This is the RSA_OAEP – RSAES step mentioned earlier. In plain English, the CEK is encrypted using asymmetric encryption. The server uses the app’s public key for this, obtained from the X509 certificate in the request.

So the way back to obtain the token plain text is to decrypt the CEK using the private key, and then decrypt the token payload using the CEK together with the IV, AAD and authTag, as illustrated in the diagram below:

jwe-decryption

That looks pretty daunting, but luckily there are libraries that help us handle this scenario, or the many other combinations of algorithms and encodings. One popular example is the JOSE-JWT library for .NET. You can feed it the complete JWE package and the CEK decryption key, and it does all the heavy lifting. JOSE-JWT can also handle our scenario, with our combination of algorithms and encodings.

Here’s the catch…

This won’t fly in a PCL. JOSE-JWT is built on System.Security.Cryptography, which isn’t available in PCL code. There have been some attempts to make JOSE-JWT available for Xamarin, but this won’t work fully cross platform. So ideally, we’d want a library that solves this using PCL compatible API’s. And sadly, there is no JOSE-JWT implementation available for PCL yet.

281H_banner.png
Now what?!

JWE just uses standard encryption algorithms, so it should be possible to implement these on top of a crypto API that is available cross platform (in PCL). I have tried two options:

  • PCLCrypto: this library mimics the WinRTCrypto API’s but relies on native,¬†platform specific crypto-engines to perform the encryption. This results in the fastest encryption engine but is limited in terms of supported scenario’s or algorithms.
  • BouncyCastle-PCL: a cross platform crypto library that supports all sorts of cryptographic algorithms.

Since PCLCrypto mimics the WinRTCrypto API, it seemed pretty easy to implement our JOSE-JWT port on top of PCLCrypto, by stealing borrowing from the experimental JOSE-RT port. (Thanks¬†Dmitriy Vsekhvalnov for the tip!)¬†And indeed, if you look at the AesGcmEncryptor class, it looks rather straight forward.¬†Alas, it turns out our GCM authenticated mode isn’t supported by PCLCrypto. It threw a NotSupportedException at me.

So the other option was to use BouncyCastle-PCL. We ended up with a solution that does the following:

  • Generation of the public/private keypair
  • Generation of the X509 certificate containing the public key
  • Unpacking the JWE package
  • Decrypting (or unwrapping) the CEK using the private key
  • Decrypting the token payload

Mind you, I only had to support our specific scenario, but it’s a small step¬†towards a full JOSE-PCL implementation. Maybe someday I’ll make an attempt ūüôā

Without further ado, here is the code I ended up with:

using System;
using System.IO;
using System.Text;
using Org.BouncyCastle.Asn1.Pkcs;
using Org.BouncyCastle.Asn1.X509;
using Org.BouncyCastle.Crypto;
using Org.BouncyCastle.Crypto.Encodings;
using Org.BouncyCastle.Crypto.Engines;
using Org.BouncyCastle.Crypto.Generators;
using Org.BouncyCastle.Crypto.Modes;
using Org.BouncyCastle.Crypto.Operators;
using Org.BouncyCastle.Crypto.Parameters;
using Org.BouncyCastle.Math;
using Org.BouncyCastle.Pkcs;
using Org.BouncyCastle.Security;
using Org.BouncyCastle.Utilities;
using Org.BouncyCastle.Utilities.IO.Pem;
using Org.BouncyCastle.X509;
using System.Linq;

namespace JoseJWE
{
	public class CryptoService
	{
		public AsymmetricCipherKeyPair GenerateKeyPair()
		{
			var random = new SecureRandom();
			var keyGenerationParameters = new KeyGenerationParameters(random, 1024);
			var keyPairGenerator = new RsaKeyPairGenerator();
			keyPairGenerator.Init(keyGenerationParameters);
			var keyPair = keyPairGenerator.GenerateKeyPair();
			return keyPair;
		}

		public string GeneratePemEncodedCertificate(AsymmetricCipherKeyPair keyPair)
		{
				var random = new SecureRandom();
				var signatureFactory = new Asn1SignatureFactory("SHA256WithRSA", keyPair.Private, random);

				var gen = new X509V3CertificateGenerator();
				gen.SetPublicKey(keyPair.Public);

				BigInteger serialNumber = BigIntegers.CreateRandomInRange(BigInteger.One, BigInteger.ValueOf(long.MaxValue), random);
				gen.SetSerialNumber(serialNumber);

				var x509Name = new X509Name("CN=MyCertificate,O=RoyCornelissen,OU=CryptoService");
				gen.SetIssuerDN(x509Name);
				gen.SetSubjectDN(x509Name);

				gen.SetNotBefore(DateTime.UtcNow.AddHours(-1));
				gen.SetNotAfter(DateTime.UtcNow.AddMonths(1));

				var x509 = gen.Generate(signatureFactory);
				x509.CheckValidity(DateTime.UtcNow);
				x509.Verify(keyPair.Public);

				using (var stringWriter = new StringWriter())
				{
					var writer = new PemWriter(stringWriter);
					var pog = new PemObject("CERTIFICATE", x509.GetEncoded());
					writer.WriteObject(pog);
					return stringWriter.ToString();
				}
		}

		public string DecodeJwt(string tokenData, AsymmetricKeyParameter privateKey)
		{
			var token = Parse(tokenData);
			return DecodeAndDecrypt(token, privateKey);
		}

		private string DecodeAndDecrypt(byte[][] parts, AsymmetricKeyParameter key)
		{
			byte[] header = parts[0];
			byte[] encryptedCek = parts[1];
			byte[] iv = parts[2];
			byte[] cipherText = parts[3];
			byte[] authTag = parts[4];

			var cek = Unwrap(encryptedCek, key);
			var aad = Encoding.UTF8.GetBytes(Serialize(header));

			return Decrypt(cek, iv, aad, cipherText, authTag);
		}

		private string Serialize(params byte[][] parts)
		{
			var builder = new StringBuilder();

			foreach (var part in parts)
			{
				builder.Append(Base64UrlEncode(part)).Append(".");
			}

			builder.Remove(builder.Length - 1, 1);

			return builder.ToString();
		}

		private byte[][] Parse(string token)
		{
			string[] parts = token.Split('.');

			var result = new byte[parts.Length][];

			for (int i = 0; i < parts.Length; i++)
			{
				result[i] = Base64UrlDecode(parts[i]);
			}

			return result;
		}

		private byte[] Unwrap(byte[] encryptedCek, AsymmetricKeyParameter key)
		{
			var decryptEngine = new OaepEncoding(new RsaEngine());
			decryptEngine.Init(false, key);
			var deciphered = decryptEngine.ProcessBlock(encryptedCek, 0, encryptedCek.Length);
			return deciphered;
		}

		//Preconfigured Encryption Parameters
		private static readonly int MacBitSize = 128;

		/// <summary>
		/// Performs AES decryption in GCM chaining mode over cipher text
		/// </summary>
		/// <param name="cek">aes key</param>
		/// <param name="iv">initialization vector</param>
		/// <param name="aad">additional authn data</param>
		/// <param name="cipherText">cipher text message to be decrypted</param>
		/// <param name="authTag">authentication tag</param>
		/// <returns>decrypted plain text messages</returns>
		private string Decrypt(byte[] cek, byte[] iv, byte[] aad, byte[] cipherText, byte[] authTag)
		{
				var keyParameter = new KeyParameter(cek);
				var gcmParameters = new AeadParameters(
					keyParameter,
					MacBitSize,
					iv);

				var gcmMode = new GcmBlockCipher(new AesFastEngine());
				gcmMode.Init(false, gcmParameters);
				gcmMode.ProcessAadBytes(aad, 0, aad.Length);

				var cipherBuffer = cipherText.Concat(authTag).ToArray();
				var plainBytes = new byte[gcmMode.GetOutputSize(cipherBuffer.Length)];
				var res = gcmMode.ProcessBytes(cipherBuffer, 0, cipherBuffer.Length, plainBytes, 0);
				gcmMode.DoFinal(plainBytes, res);

				var plain = Encoding.UTF8.GetString(plainBytes, 0, plainBytes.Length);
				return plain;
		}

		// from JWT spec
		public byte[] FromBase64Url(string base64Url)
		{
			string padded = base64Url.Length % 4 == 0
				? base64Url : base64Url + "====".Substring(base64Url.Length % 4);
			string base64 = padded.Replace("_", "/")
									.Replace("-", "+");
			return Convert.FromBase64String(base64);
		}

		// from JWT spec
		public string Base64UrlEncode(byte[] input)
		{
			var output = Convert.ToBase64String(input);
			output = output.Split('=')[0]; // Remove any trailing '='s
			output = output.Replace('+', '-'); // 62nd char of encoding
			output = output.Replace('/', '_'); // 63rd char of encoding
			return output;
		}

		// from JWT spec
		private byte[] Base64UrlDecode(string input)
		{
			var output = input;
			output = output.Replace('-', '+'); // 62nd char of encoding
			output = output.Replace('_', '/'); // 63rd char of encoding
			switch (output.Length % 4) // Pad with trailing '='s
			{
				case 0: break; // No pad chars in this case
				case 1: output += "==="; break; // Three pad chars
				case 2: output += "=="; break; // Two pad chars
				case 3: output += "="; break; // One pad char
				default: throw new Exception("Illegal base64url string!");
			}
			var converted = Convert.FromBase64String(output); // Standard base64 decoder
			return converted;
		}
	}
}

Its usage is demonstrated by the following unit tests:


<pre>using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using FluentAssertions;
using Jose;
using NUnit.Framework;
using Org.BouncyCastle.Crypto.Parameters;

namespace JoseJWE.Tests
{
	[TestFixture]
	public class CryptoServiceTests
	{
		private const string TokenPlainText = "{\"sub\":\"roycornelissen\",\"aud\":\"sample app\",\"nbf\":136424444,\"iss\":\"https://api.someorganization.com\",\"preferred_username\":\"Roy Cornelissen\",\"exp\":1364293137,\"given_name\":\"Roy\",\"iat\":13642555,\"family_name\":\"Cornelissen\",\"preferred_language\":\"nl-NL\"}";

		[Test]
		public void GenerateCertificate_Generates_Valid_X509Certificate()
		{
			var g = new CryptoService();
			var keyPair = g.GenerateKeyPair();

			var pemEncodedCertificate = g.GeneratePemEncodedCertificate(keyPair).ToString();

			var p = new Org.BouncyCastle.X509.X509CertificateParser();
			var certDecoded = p.ReadCertificate(Encoding.UTF8.GetBytes(pemEncodedCertificate));

			certDecoded.Should().NotBeNull();
			certDecoded.NotBefore.Should().BeBefore(DateTime.UtcNow);
			certDecoded.NotAfter.Should().BeAfter(DateTime.UtcNow);
		}

		[Test]
		public void Certificate_Used_For_JWT_Encryption_JWE_Can_Be_Decrypted()
		{
			var g = new CryptoService();
			var keypair = g.GenerateKeyPair();

			var cert = g.GeneratePemEncodedCertificate(keypair);
			var base64Certificate = g.Base64UrlEncode(Encoding.UTF8.GetBytes(cert));

			// try to perform local encryption and decryption for reference
			var p = new Org.BouncyCastle.X509.X509CertificateParser();
			var certDecoded = p.ReadCertificate(g.FromBase64Url(base64Certificate));

      // use 3rd party library JOSE-JWT to encode the JWT (only works in .NET, not from PCL!)
			var publicRsaKey = ToRSA((RsaKeyParameters)certDecoded.GetPublicKey());
			var encrypted = JWT.Encode(TokenPlainText, publicRsaKey, JweAlgorithm.RSA_OAEP, JweEncryption.A256GCM);

			// now attempt to decode it using our own cryptoService
			var plainText = g.DecodeJwt(encrypted, keypair.Private);
			plainText.Should().Be(TokenPlainText);
		}
    
    public static RSA ToRSA(RsaKeyParameters rsaKey)
    {
        RSAParameters rp = ToRSAParameters(rsaKey);
        RSACryptoServiceProvider rsaCsp = new RSACryptoServiceProvider();
        rsaCsp.ImportParameters(rp);
        return rsaCsp;
    }

    private static RSAParameters ToRSAParameters(RsaKeyParameters rsaKey)
    {
        RSAParameters rp = new RSAParameters();
        rp.Modulus = rsaKey.Modulus.ToByteArrayUnsigned();
        if (rsaKey.IsPrivate)
            rp.D = rsaKey.Exponent.ToByteArrayUnsigned();
        else
            rp.Exponent = rsaKey.Exponent.ToByteArrayUnsigned();
        return rp;
    }
	}
}</pre>

The entire Gist is here on GitHub.

Advertisements

There is no app! – LevelUp Mobile slides

 

On September 22nd, 2016 I presented a¬†session called “There is no app!” at the LevelUp Mobile 2016 event in Leusden. Here are the slides and video of that presentation:

[The video will be placed here shortly]

Abstract
Mobile platforms are evolving and getting richer and richer in features with every new release. The OS itself is becoming the primary interface for users to interact with, as are a new category of (wearable) devices.

What does this mean for us as app developers? Are the days of the traditional ‚Äúmobile app‚ÄĚ numbered? How do we serve our end users and optimize their mobile moment as much as possible?

The average smartphone owner has installed over 100 apps on their phone while they only use between 3 and 5 apps a day. By integrating your app more deeply into the mobile operating system can greatly increase the usage of the app. During this session we will show what you can do to integrate your apps in the Windows, iOS and Android platforms to keep your app top of mind. We’ll look at spotlight search, universal links, app indexing, Cortana integration and other APIs provided by iOS and Google Play services to engage your users in your apps. We’ll also look at new interaction models that are closer to the mobile platform: widgets, 3D-Touch, etc.

Xamarin Evolve 16 – Training Day 2

Time flies! The second day of training at Xamarin Evolve is over already and I just came back from the awesome opening party in the Darwin Lounge.

The main conference starts tomorrow. I’m honored to be on stage with some industry giants at the¬†Evolve Mobile Leaders Summit¬†track in a panel discussion about The Unique Challenges of Mobile DevOps. But that’s tomorrow!

About today: before the training, the day started at 6:15am (!) with a 5k test run with for the first Mini-Hack tomorrow morning. I spot an orange Xpirit T-shirt ūüôā

The training part was¬†another intense day from 9am-6pm, but it was a good day. I’d say the content of today was a bit better and deeper than what we learned yesterday, at least for me personally. We covered three major topics:

Securing local data
This part was all about dealing with data locally on the device, especially sensitive data in terms of privacy and security. We had a look at what the Xamarin.Auth component has to offer for local storage of sensitive data. This component (you can find it on GitHub and on Nuget) uses the platform specific facilities for storing data in a secure container, i.e. the KeyChain on iOS and a secure KeyStore inside the app’s sandbox on Android.

Be sure to get the latest 1.3.0 version of the Xamarin.Auth component though, as this is a bait-and-switch PCL library that also offers support for the Windows platform, whereas the older 1.2.x version doesn’t.

warning-stamp.png

There’s one caveat with the Xamarin.Auth component though… The KeyStore file on Android is locked with a hard coded password. The Android app in one of my previous projects was actually flagged in a security audit because they extracted the Xamarin.Auth DLL, decompiled it and found the hard coded key. Pretty iffy, because this means that the KeyStore data in every app that uses this library can be easily opened and read by extracting the file from the sandbox and using the key that’s publicly available on GitHub!

I made a pull request about 2 years ago that fixes this problem, at least in a way that you can provide your own key. But somehow Xamarin didn’t get around to merge it yet, so the vulnerability was still there, also in the 1.3.x version. The funny thing was that as we were doing the training, one of the Xamarin developers was actively working on this component. We pulled him into the training, explained the problem and he immediately went to work to see if he could merge my fix. How awesome!

FullSizeRender 4.jpg
How awesome is this: Xamarin Developer DNA… everyone could add their own string, picking the pegs that fit with their preference. The result is this awesome, unique “Xamarin DNA” sequence. A work of art!

The other part of this chapter was about the awesome PCL.Crypto component. This component is also a bait-and-switch PCL library, which means that the PCL portion contains nothing more than the API definitions for your shared code to compile against (the bait), but uses the actual device specific implementation at runtime (the switch). This means that PCL.Crypto can use the platform specific crypto API’s developed by Google, Apple and Microsoft themselves instead of relying on its own security implementation. Much the same as the Xamarin.Auth component solves its local storage issues. For developers familiar with the WinRT crypt API’s for example, there is a special WinRTCrypto API that you can program against, but PCL.Crypto will map this to the underlying native API’s. Pretty clever. For example: a method for encrypting some data¬†could look like this:

public static byte[] Encrypt (byte[] plainText, byte[] keyMaterial)
{
  var provider = WinRTCrypto.SymmetricKeyAlgorithmProvider
    .OpenAlgorithm(SymmetricAlgorithm.AesCbcPkcs7);

  var key = provider.CreateSymmetricKey(keyMaterial);
  var IV = WinRTCrypto.CryptographicBuffer.GenerateRandom(16);
  var cipher = WinRTCrypto.CryptographicEngine.Encrypt(key, plainText, IV);

  var cipherText = new byte[IV.Length + cipher.Length];
  IV.CopyTo(cipherText, 0);
  cipher.CopyTo(cipherText, IV.Length);

  return cipherText;
}

PCL.Crypto can be used quite easily in combination with Xamarin.Auth to encrypt or hash data before storing it. At least as long as your app is using the not-so-secure version of Xamarin.Auth – with the hard coded key – using something like PCL.Crypto to secure the values that go into the secure storage is a real necessity! But it’s good practice to do it anyway.

OAuth
Next we went into OAuth for authorizing access to an API from a mobile app. OAuth in itself is a pretty broad topic and enough to fill tens of blogposts. One of the important points here is that for mobile apps, basically there are only two OAuth flows that are really usable:

Screen Shot 2016-04-26 at 22.50.26

The Implicit flow or the Authorization Code flow. The other flows aren’t really suitable for mobile apps (or browser apps involving JavaScript code), since this means that there will be client secrets hidden inside the app, and they involve dealing with a user’s password in the app itself instead of handing that off to an Identity Provider. The Client Credentials flow is typically used for service-to-service communication, usually on behalf of the user.

Xamarin.Auth provides some nice API’s to easily initiate a login sequence, using a web view that follows all the redirects that are part of the OAuth flow, to obtain an access token (or authorization code).

Memory Management Issues
The most interesting part of today was about diagnosing and dealing with Memory Management Issues. I actually learned a lot about how both the Xamarin.iOS and Xamarin.Android frameworks work in terms of memory allocation. It’s important to understand that in both cases, you are always dealing with native objects, managed peers and a binding layer in between, provided by Xamarin. At least, for those objects that are actual native platform classes.

Under the hood, there are some intricacies to be aware of. For example: in order for iOS’s reference counting mechanism to work, you have to be very careful to release references to native objects, for example by making sure to always unsubscribe from event handlers. For Android, it’s important to realise that you’re working with two Garbage Collectors: the Mono one and the one in the Android Java VM. There are a lot of details, but there is some nice guidance up on the Xamarin Developer site about this [iOS] [Android].

You can prevent a lot of memory issues by following a couple of important guidelines. Also the Xamarin Profiler is a great tool for diagnosing possible memory leaks.

Lots of excitement for the opening of the Darwin Lounge at #XamarinEvolve

A post shared by Roy Cornelissen (@roycornelissen) on

 

Darwin Lounge
As for the (other) fun part of Evolve: the Darwin Lounge was opened this evening, accompanied by a huge buffet and a nice range of tasting stands for artisanal chocolate, local beers and hipster coffee ūüôā This tweet sums up how I felt this evening:

It’s no secret that I’m an avid foodie, so suffice to say that I was in seventh heaven when it comes to the food that was served here. This means that you have to sit through my Instagram food porn pictures now:

Excellent chocolate. The Phillipines one is a clear winner #food #XamarinEvolve

A post shared by Roy Cornelissen (@roycornelissen) on

 

I expected to visit a mobile dev conference but instead I've come into food discovery heaven. #XamarinEvolve #coffee

A post shared by Roy Cornelissen (@roycornelissen) on

 

Great hipster pour-over coffee at #XamarinEvolve

A post shared by Roy Cornelissen (@roycornelissen) on

 

Hmm, macarons #XamarinEvolve #food

A post shared by Roy Cornelissen (@roycornelissen) on

 

Beer tasting at #XamarinEvolve. Local beer from Orlando Brewery. Good stuff.

A post shared by Roy Cornelissen (@roycornelissen) on

 

 

So… Xamarin sure knows how to throw a party ūüôā Of course, the Darwin Lounge at Evolve is mainly about cool geek stuff and tech inspiration. There’s lots of that going on as well. Lots of IoT stuff, drones flying around, etcetera. Check out the Twitter timeline for #XamarinEvolve for a great impression of the fun things out there.

IMG_0946
Great buzz and lots of hacking going on in the Darwin Lounge
IMG_0944
These robot spiders are pretty creepy!
IMG_0943
LEGO Mindstorms, totally cool! Their official iPad app for programming the Mindstorms robot was built with Xamarin

Be sure to tune into the Evolve keynote tomorrow at 9am EST. Rumours are that it’ll be spectacular! ūüėČ

Everybody gets a Xamarin!

This week at Microsoft’s //build/ 2016 conference in San Francisco, Scott Guthrie shared more details about the Xamarin acquisition and what this means for developers on the Microsoft stack. The keynote is worth checking out, also for the great Azure content. More interesting details about what Xamarin has in the works can be found in Miguel de Icaza’s session. Or you can watch the distilled announcement here:

In short, Microsoft has made the Xamarin tools available to everyone. If you have a Visual Studio Professional or Enterprise edition, Xamarin is included at no extra cost. Moreover, it’s also available as a completely free Community Edition, under the same Microsoft license terms (small teams, OSS developers and students). Wow!

Even though the technology is¬†widely¬†regarded as excellent, as Miguel states in his talk: Xamarin used to be a bit of a niche product because of the pricing. And let’s be honest, they were steep and it turned people off, even though you could easily build a solid business case in terms of money and time saved due to higher developer productivity. But this means that Microsoft has removed a big barrier for a lot of companies and developers to adopt Xamarin! Not only that, but the fact that Microsoft is fully behind the Xamarin approach is very beneficial for customers who were betting their mobile development approach on Xamarin.

This interview with Nat Friedman on TechCrunch is an interesting read if you want to learn more about the acquisition.

Open source FTW
One of the most notable changes within Microsoft lately is their big support for open source. Microsoft emphasised this by open sourcing their .NET Framework, which has received many pull requests and active contributions since. This lead to .NET being revamped to a cross platform runtime and framework which runs on Windows, Mac and Linux as well (my colleague Alex Thissen wrote an interesting article about .NET Core in the second issue of Xpirit Magazine). This is something that Mono and Xamarin have¬†already been pioneering over the past years. It’s nice to see these things come together and the teams combining forces to move things forward. It’s a bit soon to tell what will happen with Mono in the future and whether it will merge with .NET over time, but for now it’s important to know that Mono has been re-lincensed under the MIT license, which is a pretty big deal in itself.

Moreover, Nat Friedman also announced that the whole Xamarin framework, their customized Mono runtime and Xamarin.Forms will be contributed to the OSS .NET Foundation. How cool is that!

Full cycle mobile DevOps
With the acquisition of Xamarin and – earlier – HockeyApp, Microsoft now has a pretty strong full cycle DevOps story for mobile development. Obviously the development story with cross platform C# already was strong, integration into VSTS for continuous integration is very powerful and continuous deployment to HockeyApp for internal enterprise apps is also quite easy. Check out my colleague Geert’s blog series on this topic. TestCloud remains an excellent way to mitigate test risks with their vast array of devices.

Over time, we’ll see Xamarin Insights integrated into HockeyApp for .NET native crash reporting and analytics. Xamarin Insights is a very nice product and their native support for .NET exception stack traces and ability to use the SDK¬†in¬†a shared PCL project is pretty powerful. I expect that after merging with HockeyApp, the pricing model for the analytics part will be much more attractive as well.

It would be cool to see more seamless integration of tools like Fastlane for easy and automated submission to public AppStores, etcetera.

Everyone can do Xamarin now! (?)
This all means that every .NET developer can do mobile development with Xamarin now.

Or at least, technically… Moving into mobile development – especially iOS and Android – coming from a Windows, ASP.NET, back-end development background isn’t as easy as 1-2-3.

To me the beauty of Xamarin has always been that they provide unrestricted access to every native API that is also available to Swift, Objective-C and/or Android Java developers. In my opinion, this still yields the highest quality apps while still being able to reuse a good portion of your code across platforms. The power lies in the fact that you can play with the level of reuse vs. native platform specific code as it suits you as opposed to going all in with UI abstractions and reuse all of your code. Abstraction in software development is a slippery slope and Xamarin allows you to stop abstracting if you feel it goes too far. I sincerely hope that this direct-access-to-native-API’s approach will remain a core feature of the Xamarin product, and we won’t be forced into a “UWP for all platforms” model.

Having access to all the native API’s lets you leverage the unique capabilities of the underlying OS-es. Apple Pay, Touch-ID, etcetera. But this requires deep knowledge of how these OS-es work and how to use their API’s. iOS and Android are very different beasts when it comes to their architecture and solutions for problems. This means that in my opinion, there will still be iOS developers, Android developers and Windows Developers, even though they’re all working with the same language. Sure you can master them all, but I think a good mix of specialism and multi-disciplinary teams are the way to go. Mobile devs are just like normal humans, and can be pretty biased about their platform of preference. This sometimes makes for great and interesting¬†discussions on how to solve a particular use case on all platforms, aside from the occasional fun banter about the Windows Mobile app gap, not being able to open links on iOS, or the immense device and OS fragmentation in the Android world.

Rohancharge

In any case, there will be a whole army of .NET developers coming to the Xamarin platform and there will be a huge demand for knowledge. Xamarin’s Developer Center has been revamped and it looks very nice. It’s a great resource for learning how to build apps with Xamarin and they do an excellent job explaining the native API’s and how to leverage them from C#. But over the years I have found that presenting at user groups and conferences and delivering in-person training is also a great way to share knowledge.

Both Marcel and I at Xpirit have been heavily invested in Xamarin since the early days and we’ve had the opportunity to fly around the world to speak about Xamarin. I look forward to continue sharing our knowledge. Our team was recently enforced with Geert joining Xpirit, ¬†and we’re dedicated to helping developers and customers do professional mobile development with Xamarin and Microsoft.

Come meet us and dive in head first with us at one of the following opportunities:

My hopes and expectations
It surely looks like Microsoft is invested in making the Xamarin tools a first class citizen in the .NET developer stack and Visual Studio ecosystem. We can expect deeper integration of Xamarin into Visual Studio, and Miguel already showed a glimpse of that with an iOS Simulator running on Windows. This sparked some debate about whether or not this is an actual simulator running natively on Windows, but if you listen carefully, Miguel explains that it is a simulator window remoted to a Mac. I think this trick is similar to how Xamarin does it’s build & app packaging for iOS with a remote build host over SSH:

FullSizeRender 2

I’ve seen a setup like this at many companies, where Windows PC’s were the standard developer setup, and Mac’s are considered “exotic”. Using a Mac Mini as a remote build host works ok, but if you wanted to test and debug, you needed to switch to that Mac physically or access it via VNC. That was sub-optimal and this solution is pretty sweet. ¬†I don’t think we’ll be able to have a native iOS developer story¬†without a Mac because of Apple restrictions regarding their SDK but this¬†promises to make the developer experience for a Windows based developer a whole lot smoother. I think Marcel will be happy ūüôā

It’s no secret that I’m more biased towards iOS and developing on the Mac. Using Xamarin Studio, sometimes switching to Xcode for the Interface Builder and the pleasant experience with the much less bloated Xamarin Studio IDE has become more natural to me than working from Windows and I’ve come to prefer it. With Microsoft pushing towards developer tools for Linux and Mac (VS Code), it looks like Xamarin Studio is also here to stay, and hopefully with the same level of investment to push the product ahead. Who knows, maybe Xamarin Studio¬†might become the¬†official Visual Studio IDE for the Mac one day ūüôā

In his //build/ session, Miguel showed off Xamarin Workbooks, which is based on the Xamarin Inspector. It looks very sweet as a test-bench for C# code and documentation tool. But even better: it can now be used to inspect and play with a live running app on the simulator! That is awesome as it shortens the dev/build/run/debug cycle tremendously. I hope that it will become a core part of the developer experience from within Xamarin Studio / Visual Studio but for now it has already saved me a lot of time this week while trying it out.

I also have good hopes that the original character of Xamarin’s approach (full access to native API’s) will remain intact. UWP sounds like a good idea for the Windows platform but Xamarin has always stressed that Xamarin.Forms has very specific use cases. It’s not a one-size-fits-all solution, and frankly that’s the main thing I’ve been arguing¬†against ever since I chose to go with Xamarin as opposed to Cordova, Titanium or others. So while I do expect that there will be something like a UWP for Windows, iOS¬†and Android, it cannot be the only way to develop native mobile apps.

More dots to come…
Evolve 2016 promises to be a fantastic conference and a big party to celebrate the future of Xamarin at Microsoft. I hope to see you there.

Nat Friedman wasn’t kidding when he replied to my Instagram musing:

#Xamarin timeline. Proud to have been along for the ride since even before that first dot ūüĎć

A post shared by Roy Cornelissen (@roycornelissen) on

Here’s to more dots!

GMImagePicker ported to Xamarin.iOS

TL;DR
I ported GMImagePicker to C#. Code here, Nuget here. Happy coding!

This past week I was working on a Xamarin project where we need support for selecting multiple images and/or taking pictures and uploading them to a backend service. The default UIImagePicker control in iOS is ok but not very versatile. It can take a picture with the camera, or lets you select a single image from your gallery but that’s about it. Furthermore, working with the resulting images is quite cumbersome as you’re working with large UIImage objects in memory.¬†A lot of performance and memory issues can happen if you’re not careful.

In order to deal with photos and videos more efficiently and in a much richer manner, Apple introduced the PhotoKit API in iOS 8. Mike Bluestein has written a nice introductory post on this API on the Xamarin blog, so I’m not going to repeat this.

In short, PhotoKit works with the notion of PHAsset objects, which are basically descriptors for media files on the device. There are API’s to query different photo galleries, etcetera. Only once you actually need the image for display or other purposed do you have to retrieve the image using the PHAsset descriptor. Very efficient.

Many apps, such as the Facebook app, allow users to select multiple images in a user friendly manner, and maybe even add pictures on the go by providing access to the camera while they are browsing their gallery. This is something that we also wanted to add to our app. Luckily, there are some nice open source projects around that implement just that. One of the nicest ones is the GMImagePicker component by Guillermo Muntaner Perelló, which uses the PhotoKit API under the hood. The user experience looks like this:

gmimagepickerdemo

That’s slick! You can browse through several collections, and the control is highly customizable, and even contains localized texts for labels, buttons and dialogs. Only, it’s written in Objective-C…

I had two options: bind the API using Xamarin’s Objective Sharpie or port it verbatim to C#. I chose to port it, mainly to have full control over the inner workings of the control and to not have to pull in a “foreign” language into the project. The port has complete feature parity with the Objective-C version and I tried to iron out as many issues as I could. It seems to be working pretty smoothly in my Xamarin app.

The code is up on GitHub and you can use the control by either downloading the code and including the .csproj in your project, or install the Nuget package in your Xamarin.iOS app:

Install-Package GMImagePicker.Xamarin

As I said, the GMImagePicker control is highly customizable. You can change its appearance by specifying colors for different parts of the UI, and you can provide custom titles and confirmation prompts. It’s also possible to filter and limit the types of assets you want the user to select. The whole range of options can be found in the sample app that comes with the control. Here is an overview:

var picker = new GMImagePickerController {
Title = &amp;amp;quot;Custom Title&amp;amp;quot;,
CustomDoneButtonTitle = &amp;amp;quot;Finished&amp;amp;quot;,
CustomCancelButtonTitle = &amp;amp;quot;Nope&amp;amp;quot;,
CustomNavigationBarPrompt = &amp;amp;quot;Take a new photo or select an existing one!&amp;amp;quot;,
ColsInPortrait = 3,
ColsInLandscape = 5,
MinimumInteritemSpacing = 2.0f,
DisplaySelectionInfoToolbar = true,
AllowsMultipleSelection = true,
ShowCameraButton = true,
AutoSelectCameraImages = true,
ModalPresentationStyle = UIModalPresentationStyle.Popover,
MediaTypes = new [] { PHAssetMediaType.Image },
// Other customizations to play with:
//ConfirmSingleSelection = true,
//ConfirmSingleSelectionPrompt = &amp;amp;quot;Do you want to select the image you have chosen?&amp;amp;quot;,
//PickerBackgroundColor = UIColor.Black,
//PickerTextColor = UIColor.White,
//ToolbarBarTintColor = UIColor.DarkGray,
//ToolbarTextColor = UIColor.White,
//ToolbarTintColor = UIColor.Red,
//NavigationBarBackgroundColor = UIColor.Black,
//NavigationBarTextColor = UIColor.White,
//NavigationBarTintColor = UIColor.Red,
//PickerFontName = &amp;amp;quot;Verdana&amp;amp;quot;,
//PickerBoldFontName = &amp;amp;quot;Verdana-Bold&amp;amp;quot;,
//PickerFontNormalSize = 14.0f,
//PickerFontHeaderSize = 17.0f,
//PickerStatusBarStyle = UIStatusBarStyle.LightContent,
//UseCustomFontForNavigationBar = true,
};

// You can limit which galleries are available to browse through
picker.CustomSmartCollections = new [] {
PHAssetCollectionSubtype.SmartAlbumUserLibrary,
PHAssetCollectionSubtype.AlbumRegular
};

// Event handling
picker.FinishedPickingAssets += Picker_FinishedPickingAssets;
picker.Canceled += Picker_Canceled;

// Other events to implement in order to influence selection behavior:
// Set EventArgs::Cancel flag to true in order to prevent the action from happening
picker.ShouldDeselectAsset += (s, e) =&amp;amp;gt; { /* allow deselection of (mandatory) assets */ };
picker.ShouldEnableAsset += (s, e) =&amp;amp;gt; { /* determine if a specific asset should be enabled */ };
picker.ShouldHighlightAsset += (s, e) =&amp;amp;gt; { /* determine if a specific asset should be highlighted */ };
picker.ShouldShowAsset += (s, e) =&amp;amp;gt; { /* determine if a specific asset should be displayed */ };
picker.ShouldSelectAsset += (s, e) =&amp;amp;gt; { /* determine if a specific asset can be selected */ };
picker.AssetSelected += (s, e) =&amp;amp;gt; { /* keep track of individual asset selection */ };
picker.AssetDeselected += (s, e) =&amp;amp;gt; { /* keep track of individual asset de-selection */ };

// The GMImagePicker can be treated as a PopOver as well:
var popPC = picker.PopoverPresentationController;
popPC.PermittedArrowDirections = UIPopoverArrowDirection.Any;
popPC.SourceView = gmImagePickerButton;
popPC.SourceRect = gmImagePickerButton.Bounds;

await PresentViewControllerAsync(picker, true);

The extensibility is very convenient. For example: if you want to set a maximum to the total size of the images you want to allow the user to select, you can handle the AssetSelected event, keep track of the total size selected, and handle ShouldSelectAsset, to prevent selection if a maximum threshold has been reached. This is exactly what we wanted to have in our app.

Once the user has finished selecting assets, you can use a PHImageManager to retrieve the actual images in whatever size you like:

void FinishedPickingAssets (object s, MultiAssetEventArgs e)
{
  PHImageManager imageManager = new PHImageManager();

  foreach (var asset in e.Assets) {
    imagePreview.Image = null;

    imageManager.RequestImageForAsset (asset, 
      new CGSize(asset.PixelWidth, asset.PixelHeight), 
      PHImageContentMode.Default, 
      null, 
      (image, info) => {
        // do something with the image (UIImage), e.g. upload to server
        // you can get the JPEG byte[] via image.AsJPEG()
      });
  }
}

Very nice, and it’s now available for Xamarin.iOS developers as well ūüôā

photo-1437419764061-2473afe69fc2
Photo by Andrew Illarionov (https://unsplash.com/photos/-WW8jBak7bo)

Many thanks to Guillermo for letting me port his excellent code and publish it. I’d love to hear your feedback on the code and would love to see your PR’s for improvements.

Push notifications in iOS with MonoTouch

A recurring theme when building mobile apps is push notifications. I’m working on a couple of apps at Info Support using MonoTouch for iOS and we want to add push notifications to those apps. There’s a lot of interesting and very useful information on the internet about the implementation of notifications, which is actually pretty straight forward once you know the API.

First of all, Apple has an excellent iOS Developer Library with very comprehensive and pleasant to read information about the iOS API’s. Here is the section on Notifications.

Apple makes a distinction between local and remote notifications. Local notifications being notifications you schedule and “send” from the device itself from a background task, remote notifications being the push notifications coming from the Apple Push Notification service (APN). I’m specifically looking at push notifications.

Handling notifications on the iDevice

There is a great sample implementation showing the basics of handling notifications in the iOS app here on Google Code. I shamelessly took the code snippets from that sample ūüôā In short, it’s as simple as this (in C#/MonoTouch):

// This method is invoked when the application has loaded its UI and its ready to run
public override bool FinishedLaunching (UIApplication app, NSDictionary options)
{
	//This tells our app to go ahead and ask the user for permission to use Push Notifications
	// You have to specify which types you want to ask permission for
	// Most apps just ask for them all and if they don't use one type, who cares
	UIApplication.SharedApplication.RegisterForRemoteNotificationTypes(UIRemoteNotificationType.Alert
	                                                                   | UIRemoteNotificationType.Badge
	                                                                   | UIRemoteNotificationType.Sound);

	//The NSDictionary options variable would contain our notification data if the user clicked the 'view' button on the notification
	// to launch the application.  So you could process it here.  I find it nice to have one method to process these options from the
	// FinishedLaunching, as well as the ReceivedRemoteNotification methods.
	processNotification(options, true);

	//See if the custom key value variable was set by our notification processing method
	if (!string.IsNullOrEmpty(launchWithCustomKeyValue))
	{
		//Bypass the normal view that shows when launched and go right to something else since the user
		// launched with some custom value (eg: from a remote notification's 'View' button being pressed, or from a url handler)

		//TODO: Insert your own logic here
	}

	// If you have defined a view, add it here:
	// window.AddSubview (navigationController.View);

	window.MakeKeyAndVisible ();

	return true;
}

On startup, the AppDelegate calls UIApplication.SharedApplication.RegisterForRemoteNotificationTypes() from within the FinishedLaunching method, in which you specify which notifications you are interested in. These can be a combination of UIRemoteNotificationType.Alert (for alert texts), UIRemoteNotificationType.Badge (for numbers to update the application’s badge with) and UIRemoteNotificationType.Sound (for a custom sound to be played upon receipt of the notification).

iOS will register itself with the APN and will do a callback to the application when that is finished. This can have one of two outcomes: either it succeeds or it fails. You handle this by overriding two methods in the AppDelegate: RegisteredForRemoteNotifications and FailedToRegisterForRemoteNotifications. In case of success, Apple will give you a device token with which the APN can address your application on your device. Typically, what you do in RegisteredForRemoteNotifications is passing that device token to your application services (the server application that generates the notifications) so that it can find you.

public override void RegisteredForRemoteNotifications (UIApplication application, NSData deviceToken)
{
	//The deviceToken is of interest here, this is what your push notification server needs to send out a notification
	// to the device.  So, most times you'd want to send the device Token to your servers when it has changed

	//First, get the last device token we know of
	string lastDeviceToken = NSUserDefaults.StandardUserDefaults.StringForKey("deviceToken");

	//There's probably a better way to do this
	NSString strFormat = new NSString("%@");
	NSString newDeviceToken = new NSString(MonoTouch.ObjCRuntime.Messaging.IntPtr_objc_msgSend_IntPtr_IntPtr(new MonoTouch.ObjCRuntime.Class("NSString").Handle, new MonoTouch.ObjCRuntime.Selector("stringWithFormat:").Handle, strFormat.Handle, deviceToken.Handle));

	//We only want to send the device token to the server if it hasn't changed since last time
	// no need to incur extra bandwidth by sending the device token every time
	if (!newDeviceToken.Equals(lastDeviceToken))
	{
		//TODO: Insert your own code to send the new device token to your application server
		//Save the new device token for next application launch
		NSUserDefaults.StandardUserDefaults.SetString(newDeviceToken, "deviceToken");
	}
}

public override void FailedToRegisterForRemoteNotifications (UIApplication application, NSError error)
{
	//Registering for remote notifications failed for some reason
	//This is usually due to your provisioning profiles not being properly setup in your project options
	// or not having the right mobileprovision included on your device
	// or you may not have setup your app's product id to match the mobileprovision you made
		Console.WriteLine("Failed to Register for Remote Notifications: {0}", error.LocalizedDescription);
}

Now, in the first snippet (FinishedLaunching()), you might also have noticed the call to processNotification(), which handles an incoming notification. We’ll get to the implementation of that method further on, but what’s important here is that there are two scenario’s to take care of: the first is receiving a notification while the app is running. This is handled by overriding the ReceivedRemoteNotification method in your AppDelegate. When a notification comes in, iOS will call this method when the app is active. The other scenario is when the app is not running. When iOS receives a notification and the user chooses to take action upon it, the app will launch and iOS will pass the notification data to FinishedLaunching. This is why you’ll also want to handle that from FinishedLaunching().

When a notification comes in, you basically get an NSDictionary object containing the notification data. Basically this is some JSON encoded data containing the alert, badge and name of the sound file to be played (if applicable). An alert can be a simple text string, but might also be a more complex structure, if it is a localized message. So, you process notifications from both FinishedLaunching() and ReceivedRemoteNotification(). A good practice would be to implement the handling in a separate method to which you delegate the work from both locations. So here is the implementation of processToken():

public override void ReceivedRemoteNotification (UIApplication application, NSDictionary userInfo)
{
	//This method gets called whenever the app is already running and receives a push notification
	// YOU MUST HANDLE the notifications in this case.  Apple assumes if the app is running, it takes care of everything
	// this includes setting the badge, playing a sound, etc.
	processNotification(userInfo, false);
}

void processNotification(NSDictionary options, bool fromFinishedLaunching)
{
	//Check to see if the dictionary has the aps key.  This is the notification payload you would have sent
	if (null != options && options.ContainsKey(new NSString("aps")))
	{
		//Get the aps dictionary
		NSDictionary aps = options.ObjectForKey(new NSString("aps")) as NSDictionary;

		string alert = string.Empty;
		string sound = string.Empty;
		int badge = -1;

		//Extract the alert text
		//NOTE: If you're using the simple alert by just specifying "  aps:{alert:"alert msg here"}  "
		//      this will work fine.  But if you're using a complex alert with Localization keys, etc., your "alert" object from the aps dictionary
		//      will be another NSDictionary... Basically the json gets dumped right into a NSDictionary, so keep that in mind
		if (aps.ContainsKey(new NSString("alert")))
			alert = (aps[new NSString("alert")] as NSString).ToString();

		//Extract the sound string
		if (aps.ContainsKey(new NSString("sound")))
			sound = (aps[new NSString("sound")] as NSString).ToString();

		//Extract the badge
		if (aps.ContainsKey(new NSString("badge")))
		{
			string badgeStr = (aps[new NSString("badge")] as NSObject).ToString();
			int.TryParse(badgeStr, out badge);
		}

		//If this came from the ReceivedRemoteNotification while the app was running,
		// we of course need to manually process things like the sound, badge, and alert.
		if (!fromFinishedLaunching)
		{
			//Manually set the badge in case this came from a remote notification sent while the app was open
			if (badge >= 0)
				UIApplication.SharedApplication.ApplicationIconBadgeNumber = badge;
			//Manually play the sound
			if (!string.IsNullOrEmpty(sound))
			{
				//This assumes that in your json payload you sent the sound filename (like sound.caf)
				// and that you've included it in your project directory as a Content Build type.
				var soundObj = MonoTouch.AudioToolbox.SystemSound.FromFile(sound);
				soundObj.PlaySystemSound();
			}

			//Manually show an alert
			if (!string.IsNullOrEmpty(alert))
			{
			UIAlertView avAlert = new UIAlertView("Notification", alert, null, "OK", null);
				avAlert.Show();
			}
		}
	}

	//You can also get the custom key/value pairs you may have sent in your aps (outside of the aps payload in the json)
	// This could be something like the ID of a new message that a user has seen, so you'd find the ID here and then skip displaying
	// the usual screen that shows up when the app is started, and go right to viewing the message, or something like that.
	if (null != options && options.ContainsKey(new NSString("customKeyHere")))
	{
		launchWithCustomKeyValue = (options[new NSString("customKeyHere")] as NSString).ToString();

		//You could do something with your customData that was passed in here
	}
}

Pretty nifty!

Sending notifications

In short, sending notifications to a user means that you send a notification payload to the APN using the device token to address the user. There are several ways to do that.

Christian Weyer has an excellent blog post on the implementation of push notifications using a third party for the distribution of notifications: Urban Airship. Urban Airship has a free service with which you can send up to 1,000,000 messages a month. If you want more service and features, there’s an attractive pricing model. Urban Airship abstracts the handling of multiple platforms, such as Apple, Microsoft, BlackBerry and Google, so you can service almost any type of device. PushIO offers a similar service.

If you only have to send notifications to Apple devices, since – let’s face it – iOS is the only OS that really matters… there is a nice open source C# library that does the heavy lifting for you: APNS-Sharp. This library can also handle AppStore Feedback and receipts for In App Purchases for you. Nice!

Of course, Marcel, Willem and I are targeting Android and Windows Phone as well, so a service like Urban Airship or PushIO would be ideal.

UPDATE: Added the suggestions made by Slava. Thanks for that!

GeekNight11 session online

Our “Mobile development using C#” session at GeekNight11 (at the DevDays 2011 conference) was a bit of a guerilla session. Non-Microsoft technologies, non-Microsoft platforms… a bit awkward. It worked out very nice, we got a full house (over 400 people!) and some very nice feedback. Marcel, Willem and I really enjoyed this one.

Since we were “the outlaws”, Microsoft made no official recording of the session to put on Channel9. Luckily for us, the whole session was recorded by an attendee, Tony Thijs (@tonythijs on Twitter). He used his Canon EOS500D, shooting “from the hip”. Thanks Tony!

  1. Part 1
  2. Part 2
  3. Part 3
  4. Part 4

The videos are a bit shaky and the sound is not all that, but what the heck!

Be sure to check out the DevDays sessions by my colleagues on Channel9 as well:

Full house at GeekNight... Tony is the guy in the white shirt on the left.