Debugging ARM Templates with Custom Scripts

A while back I was working a one-click Azure Deploy button for the Hoodie sample application. If you have not seen Hoodie before you should check the project out. It is an opensource project with a complete backend that allows front end developers to build amazing applications and they have a great community.

ARM Template Debugging Issues

I ran into an issue during development and testing of the ARM template that powers the one click deploy. When deploying, the template was deploying the VM successfully but the custom linux script was not completing properly.

After several attempts someone pointed out that I made a silly mistake by including a interactive prompt in the docker command. This caused one command in the script to time out but not actually fail the entire ARM deployment leaving me with no way to determine what went wrong or so I thought.

ARM Template Logs

After debugging for longer than I want to admit, I learned that the ARM templates store logs on the VM. Knowing this would have cut my debug time down considerably. You can find all the logs for ARM Deployments at:

/var/logs/Azure

In my case the log I was interested in for my custom script was located at:

/var/log/azure/Microsoft.OSTCExtensions.CustomScriptForLinux/1.5
.2.0/extension.log

Hope that helps someone not have to spend hours trying different configurations over and over because a silly interactive prompt.

Build and Deploy an opensource project with Visual Studio Team Services without sharing secrets

When working with an opensource project it is important to not keep the keys in your source control, otherwise this happens. We all know we need to keep the secrects out of public source code but how do have a Continuous Integration/Continuous Deployment setup in the case of an open source project? One solution to keep secrets out of the source code is to use envirornment varibables. At runtime you read the enviroment variables that have been set up for the given environment.

This works great for server side applications, but in the case of mobile applications where environment variables are not avalible, another strategy is to have a file that contains all of the configuration values/secrets with placeholders and then at build time either load a different file or replace the placeholders in the file for a given envirornment. Many times you might still store the envirment specific information in a environment variable and use a pre-processor to replace the placeholder with the value.

I have been working on a project with Xamarin Forms that will be released as opensource and had to figure out how to set all of this up using Visual Studio Team Services (VSTS). I have been using VSTS for the build process with a final deployment to the Hockey App. This process has been great becuase I can trigger a build using a commit into the master branch or I can trigger it manually on demand. It greatly simplifies the build process and enables me to have a repeatble process.

Step By Step Process for loading secrets in Visual Studio Team Services

This step by step does not cover how to setup your build process for Xamarin VSTS but instead covers how to setup your build process so that you can dynamically load your secrets with out having to keep them in your source. Another benifit is this can also help you provide different values for your different environments (Dev, QA, Prod, etc.).

Initial Xamarin VSTS Configuration

If you are looking for how to setup your VSTS to build Xamarin applications check out:

0. Pre-Requisits

In post I assume your VSTS configuration will already have been configured with a Repository (VSTS/Github/etc.):

image of repository configuration

And the following steps (you can refer to the links above if not):

  • NuGet Restore
  • Activate Xamarin License
  • Build Xamarin.Andriod/Xamarin.iOS
  • Signing and Provisioning Step (Andriod/iOS specific)
  • Deactivate Xamarin License
  • Publish Artifact: drop
  • Deploy to HockeyApp

image of steps already configured for VSTS

1. Find a Replace Token Task for your VSTS instance

VSTS is an extensible task based build system. VSTS has a large selection of tasks that you can use out of the box but VSTS doesn’t currently have any tasks for replacing tokens in your source control.

You could build your own custom extension, but there is also a ton of extensions on the Visual Studio Market Place contributed by community.

Searching on the Market Place for replace and you will get a few different options:

results for searching for replace on marketplace

2. Install a Replace Token Task for your VSTS instance

In this case I will use Replace Tokens task but Collins ALM Build and Release Tools has a replace token task and several other useful tasks for the build process. They function in a similiar way so it is up to you if you need a few other tasks as well.

Click Install to install the task. You will be asked to login into you VSTS account.

click install to VSTS

If you belong to multiple teams you will be given the option to install the task to a particular Team Instance of VSTS.

click install to VSTS

3. Add Replace Token Task to Build

Next we will add the new Replace Token task to our build pipeline.

Click Add Build Step.

click add build step

When the dialog opens, under the Utility tab, click Add on the Replace Tokens task.

click add on replace tokens

Finally re-arrange the task so that it is before it is before the Build.Andriod/iOS step. Since we are modifying the source code that is pulled out of the repository, we need it to be before the build step. In this case I have chosen to put it before the nuget restore so that my build fails quickly if I do not have the replace token task configured properly.

rearrange replace tokens

4. Configure Replace Token Task

After the task is in the correct place, we need to configure it to read the correct file and set the token prefix/suffix. The Root directory can be configured to to look at the entire solution or a specific folder. I perfer to keep all the configuration varaibles in a single file, so I have scoped the Root directory to a specifc folder called Constants in the UrbanRefuge.XForms project.

In the Target files option I have specified Keys.cs. You can also use wild cards when specifing the Target files option but becuase all the values are in one file I only specify the file ```Keys.cs``. You will see an example of what this file looks like in the next step.

Next up I changed the settings for Token prefix and Token suffix under the Advanced section to use a double underscore (__). This was a personal preference and can be anything you decide.

The configuration is the following:

Replace Token Task Configuration

Note: Make sure you click Save under the build tab to save configuration and the new task.

5. Setup Source Code File

Now that we have VSTS Replace Token task added to our build pipeline inside VSTS, it is time to make sure we have configured the Keys.cs file properly in our source code. This file will hold all of the placeholder tokens that the Replace Token task is going to work with. In this file each setting is a constant and uses the format __VariableName__.

namespace UrbanRefuge.XForms.Constants
{
    public class Keys
    {
        public const string AndriodHockeyAppKey = "__AndriodHockeyAppKey__";

        //Azure B2C
        public static string ClientID = "__AzureB2CClientID__";
        public static string[] Scopes = {ClientID};
        public static string SignUpSignInpolicy = "__AzureB2CSignupPolicy__";
        public static string Authority = "__AzureB2CAuthority__";
        public static string ApiUrl = "__AzureB2CApiUrl__";

        //Certs
        public static string URBAN_PUBLIC_KEY = "__UrbanRefugePublicKey__";
        public static string B2C_PUBLIC_KEY = "__B2CPublicKey__";
    }
}

This is the code that anyone browsing the opensource project on GitHub would see. There is obviously no secrets here but the place holders will be replaced when the Replace Token task runs in VSTS as part of the build and deploy to HockeyApp.

You will also notice that I have items like ApiUrl which enables me to change the URL of my API that I will be calling; Enabling me to have different versions of my app running against my Dev, QA and production environments. So this techinique is not limited to just secrets but also configuration values.

6. Add Variables in VSTS

The final step before kicking off a build is to add all the variables to VSTS. This is pretty straight forward with the option of making the variables secrets inside VSTS. By clicking the lock icon next the varaible, other team members will be able to see the varaible names but not the values. In this case, we are using the variables with the Replace Token task but this is also where you would set environment variables for applications that would need them.

add variables to build

Note: The variable names are names between the double underscores in the source and do not include the double underscrore.

7. Run Build

Finally run the build for your applicaiton and it should build and deploy to HockeyApp as a new version!

successful replace token task

Conclusion and Final thoughts

By using the extensible configuration system and a community build task we are now able to have an opensource project with out having to give up on any of the niceities that come along with a Continuous Build and Deployent practices.

Whether we use the Replace Tokens task or the Collins ALM Build and Release Tools depends on which we pefere and if we need any extra build tasks like the Version Assemblies task that Collins ALM Build and Release Tools has. Who knows there may even be other options in the Market Place when do your search.

Note: Here I have only configured the keys for some of my services. In the case of Xamarin crossplatform apps you will likely have configuration settings that are specific to each platform. For instance, for Andriod you might have configuration settings in the AndroidManifest.xml file for Maps API key. Or on iOS you might have specific configuration that needs to be changed in the Info.plist file. In this case you can add another build step and congifure it for that file specifcally using steps 3-7 as guidance (or change configuration in existing Replace Token step).

Making Http Web Requests in Unity with CSharp

This past weekend I was able to partcipate in Hack Harvard 2016 as a mentor. Hackathon are always a great time and this year there were some really great hacks ranging from solving social media bullying to auto populating calendars to a portable translator.

I got to work with lots of great teams on lots of problems but there is one that I wanted to make sure got documented. Thanks to Moaaz Elsayed who did a create write up on How to Connect Unity to Cognitive Services and Azure Table Storage.

When using Unity to make exteranl HTTP request, there are a few nuances around how you need format the requests and Moaaz did a great job summarizing a few of the challenges he resolved over the course of the weekend. It covers how to attach an image, add request headers, and adding a json body to a post request. Many other scenarios will either be a combination or derivative of those.

Be sure to check it out at:

https://github.com/moaazelsayed/UnityAzureConnection

Failed Build in Visual Studio Team Services after Xamarin upgrade

This week I updated Visual Studio’s version of Xamarin. All was well until I kicked off a build for my Andriod project in Visual Studio Team Services (VSTS). During the Build Xamarin.Android Project step I recieved the following error and my build failed:

C:\Program Files (x86)\Java\jdk1.6.0_45\\bin\javac.exe -J-Dfile.encoding=UTF8 -d obj\Release\android\bin\classes -classpath ....
obj\Release\android\src\android\support\design\R.java:10: cannot access java.lang.Object
bad class file: java\lang\Object.class(java\lang:Object.class)
class file has wrong version 52.0, should be 50.0
Please remove or make sure it appears in the correct subdirectory of the classpath.
 public final class R {

The interesting part in the error message was the class file has wrong version 52.0, should be 50.0 message. This was ultimately what helped me find the error.

After a few minutes of searching, it was obvious that I had the wrong JDK version for building my assembly. Most of the issues were Android Studio specific but I found an issue were this happened in Visual Studio. This wasn’t Visual Studio, instead VSTS but it got me down the right path.

Fix - Setting the Build Version explicitly in Build.Andriod step

Now that I had some idea of what was going on, I headed over to my VSTS Build process and noticed that there was an option in the Build Xamarin.Android step to set the JDK Version. Looking at the output above I noticed I was using JDK 6. I tried JDK 7 and got the message Unsupported major.minor version 52.0. So I tried the only value left which was JDK 8 and it worked. Here is the screen shot of the updated setting:

update jdk version number in vsts

I assume that there is a good explanation for the change but I have not found it yet. Hope this helps!

ASP.NET Core cannot find runtime for Framework '.NETCoreApp'

Today I upgrade my ASP.NET Core application from version 1.0 to version 1.0.1 because of a bug in Entity Framework. Right after updating to the latest ASP.NET Core version, I built the project but ended up with the following error in Visual Studio:

Can not find runtime target for framework '.NETFramework,Version=v4.5.1' compatible with one of the target runtimes: 'win10-x64, win81-x64, win8-x64, win7-x64'. Possible causes:
The project has not been restored or restore failed - run dotnet restore
You may be trying to publish a library, which is not supported. Use dotnet pack to distribute libraries.

Fix - Update project.json

After searching around for a few minutes I found issue #2442 on GitHub. This the issue states that you need to update your project.json and you have two options:

(1). Include the platforms you want to build for explicitly:

"runtimes": {
    "win10-x64": {},
    "win8-x64": {} 
},

(2). Update the reference Microsoft.NETCore.App to include the type as platform:

"Microsoft.NETCore.App": {
    "version": "1.0.1",
    "type": "platform"
}

Conclusion

For more information on .NET Core Application Deployment you can read the docs. This is again another reason I love that all this work is being done out in the open. It really makes finding issues and bugs of this type easier. Hope that Helps!