Deploying Scala code on Azure Functions

 

Interested in running Scala on Azure Functions?? Does Scala run better on Functions than Java? This article will outline how to publish your Scala code to Azure Functions and create a local Azure Functions development environment. + (benchmark below) 

The complete source code for this project is located here.

New 2.0 Runtime

For those of us who work frequently with Azure Functions one thing has been very clear over the few years: Microsoft is investing heavily into Functions, and the changes so far in the 2.0 runtime have made a huge performance impact. Additionally, Microsoft has worked hard to make the deployment to functions easy for “first-class” supported languages like Javascript, Java, C# and as of recently, Python. Unfortunately, there is no built in build and deploy mechanism for Scala yet, but Scala JARs execute on the Function worker with no issue.

Functions Structure

Before digging deep, it’s important to call out the structure of Functions and Function Apps in Azure.

The Function service has two entities: the Function App, and the Function. The Function App is the the collection of individual Functions and is an instance on the Azure Functions Service host. All Functions under the Function App share the same resources within an instance and scale at the time. Function Apps (remember, the collection of Functions) scale independently on the consumption plan of compute allocation.1 This is an important thing to consider when architecting how you deploy your Functions.

A Function App has the following directory structure (our folder names are used here as an example):

wwwroot
— scala-function-app 
—-+ host.json
–ScalaFunction
—-+ function.json
—-+ ScalaFunction.jar

This displays the following view from the Azure portal

Function Configuration Files

These files are necessary to deploy Functions to Azure.

  1. host.json  – This file is the global configuration metadata for all Functions (the Function App). At a minimum, it must specify the runtime version.
  2. function.json  – This is the individual Function configuration metadata. In the Java worker, there are important parameters:
    1.   scriptFile – Location of the JAR, relative to the Function App directory
    2.   entryPoint – Location of method to execute within your application. In this case, this is  Function.run

1. Create the Function Development Environment

  1. Make sure you have JDK 8 installed and your JAVA_HOME set appropriately.
  2. Install .NET Core (needed for the local function host)
  3. Install the Azure Function Core Tools (this will help you run and debug your function locally before deploying)
  4. Install sbt
  5. Clone the Scala example repo from Github

sbt Configuration

sbt needs two important configuration items:

+build.sbt

This adds the Functions Java library to the project and sets our .jar output directory to the local function directory

+plugins.sbt

The Java function worker executes your function via a packaged .JAR. We use the sbt-assembly plugin to create a fat jar with all dependencies included. This JAR will later be uploaded to Azure via the Function Core Tools

2. Converting the Java Function Code to Scala

Most of the time, Java libraries can easily be interacted with in Scala.

The Microsoft documentation here shows an example Java Function class as below:

Converting this code to Scala, we now have the following Scala Class

In this Class, the annotations are very important. Without @FunctionName()  and the @HttpTrigger()  the function will not run. The @FunctionName()  must match the function folder name as shown above. Failure to do so will show the following error when executing the function locally or in Azure:

See the complete conversion in the repo.

3. Build the JAR

The project contains a directory named ‘scala-function-app’. This is our top level Function App directory that will be deployed to Azure. We instruct sbt to deploy our JAR to this directory.

Package the jar with the command  sbt assembly .

4. Test the Function App locally

The Function Core Tools allow you to launch an instance of the Function Host locally on your machine and test your Function. The ‘scala-function-app’ directory

  1. ‘cd’ to the scala-function-app directory within the project
  2. Run the command ‘func host start’

You should see the following output:

Execute a POST with any body message to the shown URL to test your function.

4. Publish the Function App to Azure

  1. Create your Function App – either via Azure CLI or the Portal. Don’t worry about the precreated Functions or configuration files, our deploy will overwrite everything.
  2. Run the command  func azure functionapp publish {function_app_name_in_Azure} This will upload your JAR and configuration files to Azure and deploy your Function App.

That’s it! Your Function App should now be visible via the Azure portal.

Function Benchmarks

Light load testing shows Scala performs slightly (20%) faster than Java when executing a Fibonacci calculation of 20. I understand this may not be the perfect benchmark of a Function, but it definitely rules out Scala being slower than Java on Functions.

100 concurrent user load for 2 minutes was sent to the Java and Scala function at different times:

Java Function benchmark

Scala Function benchmark

Feedback appreciated in the comments below.

 

 

 

A Complete Integration – Azure AD B2C & Azure AD (Graph API, Logic Apps)

Login with Facebook, Twitter, LinkedIn or Azure AD? ” A guide outlining how to integrate Azure AD with B2C, using Logic Apps REST API & the Microsoft Graph API to retrieve user attributes. 

Disclaimer: Azure Logic Applications do not do well under heavy load. See the limits here from Microsoft before you implement this in a production environment. Generally speaking, the activity of the Logic App will be better performing on an App Service Plan or Azure Function. 

Azure AD B2C is an amazing offering from Microsoft that allows SaaS providers to offload their identity service to the cloud. Traditionally used for scenarios where integration with a social identity provider is desired, B2C whilst using the Identity Experience Framework (“custom policies / advanced policies”) can support the integration of any OAuth/OpenID Connect or SAML 2.0 Identity Provider, including Azure AD.

Why Azure AD B2C?

The business use cases for this are solid. With more than 8 million tenants in Azure AD, a company could enable “Sign in with Azure AD” and support SSO with arguably most enterprise organizations.

Integrating Azure AD & B2C has been documented by Microsoft, but there are some important gaps.

  1. B2C will only retrieve the ‘id_token’ from Azure AD, no ‘access_token’
  2. The ‘id_token’  will only contain the standard set of claim types listed here. This means our application will not be aware of claims such as title, mobilePhone, email address, or any custom attribute that we wish to deliver to the relying party (application).

Microsoft Graph & Azure Logic Apps

The solution to these gaps is proving a REST API that B2C will interact with to retrieve the user attributes.

General summary –

  1. User clicks “Login” to application
  2. B2C sends user to Azure AD Sign in page
  3. User Signs in
  4. Azure AD sends id_token to B2C (with objectid of user)
  5. B2C sends the ObjectID to the Logic App
  6. Logic app looks up user via the Graph API, sends rest of attributes to B2C
  7. B2C sends entire claim to relying party, either via SAML or JWT (OpenIDConnect)

Getting Started

Before getting started, you should have a complete B2C tenant, a POC application, and sign in with Azure AD already configured. A guide to this is provided by Microsoft here.

Microsoft Graph Application Registration & Authorization

In order to query the Graph API, we need to register a Microsoft Graph application. This is done via https://apps.dev.microsoft.com.

  1. Create an app
  2. Record your client_secret (application id)
  3. Generate and record your application secret
  4. Add Platforms > Web > Configure “http://localhost” as the redirect URL.
  5. Add Application Permission “User.Read.All”
  6. Save

Now that the app is registered, you must authorize it to query against your directory. B2C will only be sending the Object ID and tenant to the application, no access_token, so the application needs to have pre-authorization.

  1. Construct a request URL

https://login.microsoftonline.com/<tenant name>/adminconsent?client_id=<application_id>&redirect_uri=http://localhost/#

Tenant name = The FQDN of your Azur AD tenant (contoso.onmicrosoft.com)

Application_id = The complete Application ID from the apps.dev.microsoft.com site

  1. Browser to this URL in your browser
  2. Sign in with an adminstrative account of the Azure AD tenant. 
  3. Approve the application
  4. Acknowledge – You’ll be redirected to http://localhost. If the request was successful, you should see a URL  like http://localhost/?admin_consent=True&tenant=6660b467-add3-4d9e-8ada-48b1721959f3

Once you have the app authorized, the application ID and application secret, click the “Deploy to Azure” button below which will configure the Logic App immediately ready for use in your tenant.


Once the app has deployed, get the request URL for the application from within the Azure Portal. This will later be used in your B2C policy.

Integrate Logic App with B2C

Now that the Logic App is deployed, we need to add the Claim Provider and add register the Claim Provider as part of the User Journey. Also, The logic app is developed with multi-tenancy in mind, meaning we can accept requests that come in from any Azure AD tenant. However, the Logic App must know the tenant that is being used so we configure a DefaultValue claim for the Azure AD technical profile to be sent in the claim to the Logic App.

  1. Open your TrustFrameworkBase.xml

Find the tag  <ClaimsSchema>

2. Add the following code snip

3. Open your TrustFrameworkExtensions.xml file

Locate the TechnicalProfile for your Azure AD configuration. Add the following to your list of  <OutputClaims>

Replace tenant-name  with the FQDN of your tenant.

4. Register the technical profile of the Logic App

Immediately after the ending <\TechnicalProfile> of the Azure AD configuration, add the following snipit.

note: You must format your URL to be XML compliant. After each ‘&’ symbol you must add ‘amp;’

Example –

 

5. Add the Logic App to the User Journey

Locate your default UserJourney. This may be in your TrustFrameworkExtensions or Base file. Locate where your orchestration step is being executed to call against Azure AD. Immediately after, add the following code. 

Be certain to change the OrchestrationStep Order number.

 

6. Save & Upload your policies. Viola! 

 

Concluding notes

  1. Security – Integrating the Logic App, we rely on the SAS (Shared Access Signature) of the Logic App in the URL as a method of authentication. This is not the most secure approach, as anyone with the URL could query our Graph API. It is recommended if this is implemented in production, that you use Azure API Management as a wrapper around the Logic App to provide authentication and optionally caching.
  2. This example only outlines retrieving the email of the user from the Graph API. The complete list of attributes are located here. https://developer.microsoft.com/en-us/graph/docs/api-reference/v1.0/api/user_get. Make certain to add the claim to your schema before configuring an output claim of the Logic App.

PowerShell – OAuth & Downloading,Uploading to Google Drive via Drive API

Google offers a REST API that can be accessed via PowerShell to Upload, Download, and change files relatively easily.

Where does this come handy? This could be used to dynamically process Active Directory, Office 365, employee onboarding workflows, or any other system changes off a Google Sheet. This also allows the sysadmin to routinely upload logs, backup files, or any other data to Google Drive as apart of their job process.

This guide shows from start to finish how to generate an OAuth token, Download a file, and upload a file to Google Drive via the Drive REST HTTP API.

Create Project and OAuth Tokens

  1. Go to https://console.developers.google.com/iam-admin/projects?
  2. Create a new project
  3. Enable the Drive API by selecting the Drive API and selecting enable API , after creating your project.
  4. Click Credentials
  5. Create OAuth Client ID Credentials
  6. Select Web Application as product type
  7. Configure the Authorized Redirect URI to https://developers.google.com/oauthplayground must not have a ending “/” in the URI
  8. Save your client ID and Secret
  9. Browse to https://developers.google.com/oauthplayground
    This saves us time from generating an OAuth request in PowerShell. This is where we will authorize our Project to our Google Account using our provided Client ID and Client Secret. 
  10. Click the gear in the right-hand corner and select “Use your own OAuth credentials
  11. Authorize the https://www.googleapis.com/auth/drive API
  12. Click “Exchange Refresh Token for Access Token”
    explanation – Access Tokens have a limited lifetime (approximately 60 minutes) whereas Refresh Tokens last indefinitely (some exclusions here). The Access Token is what you will hardcode into your script, configuring the script to hit the Google Identity Platform to request a Refresh Token on execution. 
  13. Save your Refresh Token

Get Auth Token & Download

Now that we have our Refresh Token, Client ID, Client Secret we can request an Access Token with PowerShell.  Random but lookalike values gave for variables.

note that Invoke-RestMethod was used instead of Invoke-WebRequest. Invoke-RestMethod returns the output to a JSON object that we can easily work with going forward.

Our access token is now  $accessToken = $GAuthResponse.access_token .

Now that we have the Access Token, downloading from Drive is pretty straight forward.

Notice here that I am requesting to download a Google Sheet as CSV. This is done by calling /{DocumentID}/export?mimeType=txt/csv against the API endpoint. Other formats are supported depending on your file type, or /export can removed if the file is not a G Suite file type. More info on this here.

Uploading to Drive

Traditionally, uploads to Drive are sent via 2 requests. One to send file metadata to get the DocumentID, the second to PUT the contents of the file to the DocumentID. However, the  https://www.googleapis.com/upload/drive/v2/files endpoint supports Multi part uploads, allowing us to send metadata and file contents in one request.

 

 

 

PowerShell – Cleaning up old files with Compress-Archive & organizing by date

Windows Management Framework 5.0 introduces a few new Commandlets, notibly Compress-Archive which makes creating a .ZIP from within PowerShell extremely easy.  Below is a small project I worked on this week to clean up an FTP server used by automated processes.

This script grabs all files older than 24 hours, moves them into a folder by day, zips that folder, then deletes the folder.  Included error handling will capture the error and send via SMTP.

Any reccomendations/comments welcomed!

 

How adding digital signatures (DKIM+DMARC) to your organization’s emails might be the easiest thing you can do to prevent an attack

Note – This is an oversimplified guide to just one concept of email security. There are much more secure and advanced methods for ensuring confidentiality and integrity than DKIM available for implementation. 

Email is just like a letter from your grandma.

Many people fail to understand how utterly insecure the email standard is by default. The best way to understand email security is to compare it to a USPS First Class Mail Letter. 

A sample USPS letter

Continue reading “How adding digital signatures (DKIM+DMARC) to your organization’s emails might be the easiest thing you can do to prevent an attack”