Ok Google Email me the status of all vms – Part 1

Technology is evolving at a breathtaking pace. For instance, the phone in your pocket has more grunt than the desktop computers of 10 years ago!

One of the upcoming areas in Computing Science is Artificial Intelligence. What seemed science fiction in the days of Isaac Asimov, when he penned I, Robot seems closer to reality now.

Lately the market is popping up with virtual assistants from the likes of Apple, Amazon and Google. These are “bots” that use Artificial Intelligence to help us with our daily lives, from telling us about the weather, to reminding us about our shopping lists or letting us know when our next train will be arriving. I still remember my first virtual assistant Prody Parrot, which hardly did much when you compare it to Siri, Alexa or Google Assistant.

I decided to test drive one of these virtual assistants, and so purchased a Google Home. First impressions, it is an awesome device with a lot of good things going for it. If only it came with a rechargeable battery instead of a wall charger, it would have been even more awesome. Well maybe in the next version (Google here’s a tip for your next version 😉 )

Having played with Google Home for a bit, I decided to look at ways of integrating it with Azure, and I was pleasantly surprised.

In this two-part blog, I will show you how you can use Google Home to send an email with the status of all your Azure virtual machines. This functionality can be extended to stop or start all virtual machines, however I would caution against NOT doing this in your production environment, incase you turn off some machine that is running critical workloads.

In this first blog post, we will setup the backend systems to achieve the tasks and in the next blog post, we will connect it to Google Home.

The diagram below shows how we will achieve what we have set out to do.

Google Home Workflow

Below is a list of tasks that will happen

  1. Google Home will trigger when we say “Ok Google email me the status of all vms”
  2. As Google Home uses Google Assistant, it will pass the request to the IFTTT service
  3. IFTTT will then trigger the webhooks service to call a webhook url attached to an Azure Automation Runbook
  4. A job for the specified runbook will then be queued up in Azure Automation.
  5. The runbook job will then run, and obtain a status of all vms.
  6. The output will be emailed to the designated recipient

Ok, enough talking 😉 lets start cracking.

1. Create an Azure AD Service Principal Account

In order to run our Azure Automation runbook, we need to create a security object for it to run under. This security object provides permissions to access the azure resources. For our purposes, we will be using a service principal account.

Assuming you have already installed the Azure PowerShell module, run the following in a PowerShell session to login to Azure

Import-Module AzureRm
Login-AzureRmAccount

Next, to create an Azure AD Application, run the following command

$adApp = New-AzureRmADApplication -DisplayName "DisplayName" -HomePage "HomePage" -IdentifierUris "http://IdentifierUri" -Password "Password"

where

DisplayName is the display name for your AD Application eg “Google Home Automation”

HomePage is the home page for your application eg http://googlehome (or you can ignore the -HomePage parameter as it is optional)

IdentifierUri is the URI that identifies the application eg http://googleHomeAutomation

Password is the password you will give the service principal account

Now, lets create the service principle for the Azure AD Application

New-AzureRmADServicePrincipal -ApplicationId $adApp.ApplicationId

Next, we will give the service principal account read access to the Azure tenant. If you need something more restrictive, please find the appropriate role from https://docs.microsoft.com/en-gb/azure/active-directory/role-based-access-built-in-roles

New-AzureRmRoleAssignment -RoleDefinitionName Reader -ServicePrincipalName $adApp.ApplicationId

Great, the service principal account is now ready. The username for your service principal is actually the ApplicationId suffixed by your Azure AD domain name. To get the Application ID run the following by providing the identifierUri that was supplied when creating it above

Get-AzureRmADApplication -IdentifierUri {identifierUri}

Just to be pedantic, lets check to ensure we can login to Azure using the newly created service principal account and the password. To test, run the following commands (when prompted, supply the username for the service principal account and the password that was set when it was created above)

$cred = Get-Credential 
Login-AzureRmAccount -Credential $cred -ServicePrincipal -TenantId {TenantId}

where Tenantid is your Azure Tenant’s ID

If everything was setup properly, you should now be logged in using the service principal account.

2. Create an Azure Automation Account

Next, we need an Azure Automation account.

2.1   Login to the Azure Portal and then click New

AzureMarketPlace_New

2.2   Then type Automation and click search. From the results click the following.

AzureMarketPlace_ResultsAutomation

2.3   In the next screen, click Create

2.4   Next, fill in the appropriate details and click Create

AutomationAccount_Details

3. Create a SendGrid Account

Unfortunately Azure doesn’t provide relay servers that can be used by scripts to email out. Instead you have to either use EOP (Exchange Online Protection) servers or SendGrid to achieve this. SendGrid is an Email Delivery Service that Azure provides, and you need to create an account to use it. For our purposes, we will use the free tier, which allows the delivery of 2500 emails per month, which is plenty for us.

3.1   In the Azure Portal, click New

AzureMarketPlace_New

3.2   Then search for SendGrid in the marketplace and click on the following result. Next click Create

AzureMarketPlace_ResultsSendGrid

3.3   In the next screen, for the pricing tier, select the free tier and then fill in the required details and click Create.

SendGridAccount_Details

4. Configure the Automation Account

Inside the Automation Account, we will be creating a Runbook that will contain our PowerShell script that will do all the work. The script will be using the Service Principal and SendGrid accounts. To ensure we don’t expose their credentials inside the PowerShell script, we will store them in the Automation Account under Credentials, and then access them from inside our PowerShell script.

4.1   Go into the Automation Account that you had created.

4.2   Under Shared Resource click Credentials

AutomationAccount_Credentials

4.3    Click on Add a credential and then fill in the details for the Service Principal account. Then click Create

Credentials_Details

4.4   Repeat step 4.3 above to add the SendGrid account

4.5   Now that the Credentials have been stored, under Process Automation click Runbooks

Automation_Runbooks

Then click Add a runbook and in the next screen click Create a new runbook

4.6   Give the runbook an appropriate name. Change the Runbook Type to PowerShell. Click Create

Runbook_Details

4.7   Once the Runbook has been created, paste the following script inside it, click on Save and then click on Publish

Import-Module Azure
$cred = Get-AutomationPSCredential -Name 'Service Principal account'
$mailerCred = Get-AutomationPSCredential -Name 'SendGrid account'

Login-AzureRmAccount -Credential $cred -ServicePrincipal -TenantID {tenantId}

$outputFile = $env:TEMP+ "\AzureVmStatus.html"
$vmarray = @()

#Get a list of all vms 
Write-Output "Getting a list of all VMs"
$vms = Get-AzureRmVM
$total_vms = $vms.count
Write-Output "Done. VMs Found $total_vms"

$index = 0
# Add info about VM's to the array
foreach ($vm in $vms){ 
 $index++
 Write-Output "Processing VM $index/$total_vms"
 # Get VM Status
 $vmstatus = Get-AzurermVM -Name $vm.Name -ResourceGroupName $vm.ResourceGroupName -Status

# Add values to the array:
 $vmarray += New-Object PSObject -Property ([ordered]@{
 ResourceGroupName=$vm.ResourceGroupName
 Name=$vm.Name
 OSType=$vm.StorageProfile.OSDisk.OSType
 PowerState=(get-culture).TextInfo.ToTitleCase(($vmstatus.statuses)[1].code.split("/")[1])
 })
}
$vmarray | Sort-Object PowerState,OSType -Desc

Write-Output "Converting Output to HTML" 
$vmarray | Sort-Object PowerState,OSType -Desc | ConvertTo-Html | Out-File $outputFile
Write-Output "Converted"

$fromAddr = "senderEmailAddress"
$toAddr = "recipientEmailAddress"
$subject = "Azure VM Status as at " + (Get-Date).toString()
$smtpServer = "smtp.sendgrid.net"

Write-Output "Sending Email to $toAddr using server $smtpServer"
Send-MailMessage -Credential $mailerCred -From $fromAddr -To $toAddr -Subject $subject -Attachments $outputFile -SmtpServer $smtpServer -UseSsl
Write-Output "Email Sent"

where

  • ‘Service Principal Account’ and ‘SendGrid Account’ are the names of the credentials that were created in the Automation Account (include the ‘ ‘ around the name)
  • senderEmailAddress is the email address that the email will show it came from. Keep the domain of the email address same as your Azure domain
  • recipientEmailAddress is the email address of the recipient who will receive the list of vms

4.8   Next, we will create a Webhook. A webhook is a special URL that will allow us to execute the above script without logging into the Azure Portal. Treat the webhook URL like a password since whoever possesses the webhook can execute the runbook without needing to provide any credentials.

Open the runbook that was just created and from the top menu click on Webhook

Webhook_menu

4.9   In the next screen click Create new webhook

4.10  A security message will be displayed informing that once the webhook has been created, the URL will not be shown anywhere in the Azure Portal. IT IS EXTREMELY IMPORTANT THAT YOU COPY THE WEBHOOK URL BEFORE PRESSING THE OK BUTTON.

Enter a name for the webhook and when you want the webhook to expire. Copy the webhook URL and paste it somewhere safe. Then click OK.

Once the webhook has expired, you can’t use it to trigger the runbook, however before it expires, you can change the expiry date. For security reasons, it is recommended that you don’t keep the webhook alive for a long period of time.

Webhook_details

Thats it folks! The stage has been set and we have successfully configured the backend systems to handle our task. Give yourselves a big pat on the back.

Follow me to the next blog, where we will use the above with IFTTT, to bring it all together so that when we say “OK Google, email me the status of all vms”, an email is sent out to us with the status of all the vms 😉

I will see you in Part 2 of this blog. Ciao 😉

Advertisements

Building a VPN Solution using OpenVPN Server on a Raspberry Pi – Part 2 Configuring the clients

Now that we have configured the openvpn server, we will continue on to configuring the clients. If you haven’t configured the openvpn server yet, please refer to my previous blog https://nivleshc.wordpress.com/2017/08/31/building-a-vpn-solution-using-openvpn-server-on-a-raspberry-pi-part1-configuring-the-server/

To connect to the openvpn server, the client needs a ovpn file. This file contains the required certificates and the openvpn servers address. The article at https://readwrite.com/2014/04/11/building-a-raspberry-pi-vpn-part-two-creating-an-encrypted-client-side/#awesm=~oB89WBfWrt21bV provides the steps required. However I have deviated from the steps listed in that article abit, for the following reasons

  • I am not using comp-lzo, which is used for backwards compatibility
  • I am using AES-256-CBC
  • I am using a more verbose level (verb 3)

Now that the above is out of the way, use the steps below to generate the client configuration file

  1. On the Raspberry Pi, using terminal, elevate your session to root and then change to the easy-rsa keys folder using the following command
   sudo su
   cd /etc/openvpn/easy-rsa/keys

2.     Create a new file with the following lines and save it as Default.txt ( do not forget to replace <your public domain name to openvpn server> with your openvpn public hostname)

   client
   dev tun
   proto udp
   remote <your public domain name to openvpn server> 1194
   resolv-retry infinite
   nobind
   persist-key
   persist-tun
   mute-replay-warnings
   ns-cert-type server
   key-direction 1
   cipher AES-256-CBC
   verb 3
   mute 20

3.   Download the script MakeOpenVPN.sh from Github using the following command (the actual Github url is https://gist.github.com/laurenorsini/10013430. The command below is using the raw version of the file)

  wget https://gist.githubusercontent.com/laurenorsini/10013430/raw/df70eae7b573aaa16c417bc54c2e0c03303501e6/MakeOpenVPN.sh

4.   The MakeOpenVPN.sh script needs some modification to match our openvpn server configuration. Modify the script based on the following

change line 7 from KEY=".3des.key" to KEY=".aes256.key"
change line 21 from echo "Client’s cert found: $NAME$CR" to 
echo "Client’s cert found: $NAME$CRT" 
change line 72 by adding a # in front of the line (so you are commenting out the line)

5. The script will be owned by root. Grant it execute permissions by running the following

  chmod u+rwx MakeOpenVPN.sh

6.  Now execute the script using the following command. You will be asked for the name of an existing client for whom you had generated the keys and certificates for when configuring the openvpn server (for instance client1)

 ./MakeOpenVPN.sh

The script checks to ensure the following files exist in the folder /etc/openvpn/easy-rsa/keys

{client1}.crt
{client1}.aes256.key
ca.crt
ta.key

If any of the above files are missing, an error is displayed and the script stops.

However, if everything goes well, the following is displayed

Done! {client1}.ovpn Successfully Created.

and the file {client1}.ovpn is placed in the /etc/openvpn/easy-rsa/keys folder

7.   Thats it folks! The client config file is now ready. Import it into your openvpn client to connect to your openvpn server. Don’t forget the password you had configured for the client when you generated the key because it will be requested everytime you try connecting to the openvpn server.

A good MacOS OpenVPN client is https://tunnelblick.net. On IOS, you can download the  free OpenVPN Connect app.

Let me know what you think of my blog and enjoy the privacy and benefits of the vpn server 🙂

Script to shutdown servers

I run a lot of Microsoft virtual machines in Azure and also locally on my MacBook Pro. These are my lab machines, which I use for testing.

One of the issues with having many virtual machines is orderly shutting them down. It can be a pain to go through each of them and shutting them down.

To circumvent this, I wrote a small PowerShell script, which does it all for me 🙂

The script requires the following

$serverlist contains the hostnames of the servers that you want to shutdown (in the order they need to be shutdown)

$server_domainname this is the domain name that the servers are part of.

servername and and server_domainname is used to figure out the server fqdn, which is then used to shutdown that server.

Run the script from a computer that can connect to the servers. Ensure you are logged on as an account that has permissions to shutdown the servers.

The script will go through the list of servers contained in $serverlist and check if they are online. If they are online, then it will try to shut them down.

Do note that these servers will be forced to shutdown, so anything open on those servers will be lost, if not saved.

Once all the online servers have been shutdown, you will be asked if you want to shutdown the computer you are running the script from. You can press Enter to continue or CTRL+C to skip shutting down the computer you are logged on.

Hope this script comes in handy to others

Re-execute the UserData script in an AWS Windows Instance

Bootstrapping is an awesome way of customising your instances in AWS (similar capability exists in Azure).

To enable bootstrapping, while configuring the launch instance, in Step 3: Configure Instance Details scroll down to the bottom and then expand Advanced Details.

You will notice a User data text box. This is where you can provide your bootstrap script. The script will be run when your instance is first launched.

AWS_BootstrapScript

I went ahead and entered my script in the text box and proceeded to complete my instance configuration. Once my instance was running, I initiated a Remote Desktop connection to it, to confirm that my script had run. Unfortunately, I couldn’t see any customisations (which meant my script didn’t run)

Thinking that the instance had not been able to access the user data, I opened up Internet Explorer and then browsed to the following url (this is an internal url that can be used to access the user-data)

http://169.254.169.254/latest/user-data/

I was able to successfully access the user-data, which meant that there were no issues with that.  However when checking the content, I noticed a typo! Aha, that was the reason why my customisations didn’t happen.

Unfortunately, according to AWS, user-data is only executed during launch (for those that would like to read, here is the official AWS documentation). To get the fixed bootstrap script to run, I would have to terminate my instance and launch a new one with the corrected script (I tried re-booting my windows instance after correcting my typo, however it didn’t run).

I wasn’t very happy on terminating my current instance and then launching a new one, since for those that might not be aware, AWS EC2 compute charges are rounded up to the next hour. Which means that if I terminated my current instance and launched a new one, I would be charged for 2 x 1hour sessions instead of just 1 x 1 hour!

So I set about trying to find another solution. And guess what, I did find it 🙂

Reading through the volumes of documentation on AWS, I found that when Windows Instances are provisioned, the service that does the customisations using user-data is called EC2Config. This service runs the initial startup tasks when the instance is first started and then disables them. HOWEVER, there is a way to re-enable the startup tasks later on 🙂 Here is the document that gives more information on EC2Config.

The Amazon Windows AMIs include a utility called EC2ConfigService Settings. This allows you to configure EC2Config to execute the user-data on next service startup. The utility can be found under All Programs (or you can search for it).

AWS_EC2ConfigSettings_AllApps

AWS_EC2ConfigSettings_Search

Once Open, under General you will see the following option

Enable UserData execution for next service start (automatically enabled at Sysprep) eg. or <powershell></powershell>

AWS_EC2ConfigSettings

Tick this option and then press OK. Then restart your Windows Instance.

After your Windows Instance restarts, EC2Config will execute the userData (bootstrap script) and then it will automatically remove the tick from the above option so that the userData is not executed on subsequent restarts (or service starts)

There you go. A simple way to re-run your bootstrap scripts on an AWS Windows Instance without having to terminate the current instance and launching a new one.

There are other options available in the EC2ConfigService Settings that you can explore as well 🙂

 

Automate Secondary ADFS Node Installation and Configuration

Introduction

Additional nodes in an ADFS farm are required to provide redundancy incase your primary ADFS node goes offline. This ensures your ADFS service is still up and servicing all incoming requests. Additional nodes also help in load balancing the incoming traffic, which provides a better user experience in cases of high authentication traffic.

Overview

Once an ADFS farm has been created, adding additional nodes is quite simple and mostly relies on the same concepts for creating the ADFS farm. I would suggest reading my previous blog Automate ADFS Farm Installation and Configuration as some of the steps we will use in this blog were documented in it.

In this blog, I will show how to automatically provision a secondary ADFS node to an existing ADFS farm. The learnings in this blog can be easily used to deploy more ADFS nodes automatically, if needed.

Install ADFS Role

After provisioning a new Azure virtual machine, we need to install the Active Directory Federation Services role on it.  To do this, we will use the same Desired State Configuration (DSE) script that was used in Automate ADFS Farm Installation and Configuration. Please refer to the section Install ADFS Role in the above blog for the steps to create the DSE script file InstallADFS.ps1.

Add to an existing ADFS Farm

Once the ADFS role has been installed on the virtual machine, we will create a Custom Script Extension (CSE) to add it to the ADFS farm.

In order to do this, we need the following

  • certificate that was used to create the ADFS farm
  • ADFS service account username and password that was used to create the ADFS farm

Once the above prerequisites has been met, we need a method for making the files available to the CSE. I documented a neat trick to “sneak-in” the certificate and password files onto the virtual machine by using Desired State Configuration (DSE) package files in my previous blog. Please refer to Automate ADFS Farm Installation and Configuration under the section Create ADFS Farm for the steps.

Also note, for adding the node to the adfs farm, the domain user credentials are not required. The certificate file will be named adfs_certificate.pfx  and the file containing the encrypted adfs service account password will be named adfspass.key.

Assuming that the prerequisites have been satisfied, and the files have been “sneaked” onto the virtual machine, lets proceed to creating the CSE.

Open Windows Powershell ISE and paste the following.

param (
  $DomainName,
  $PrimaryADFSServer,
  $AdfsSvcUsername
)

The above shows the parameters that need to be passed to the CSE where

$DomainName is the name of the Active Directory domain
$PrimaryADFSServer is the hostname of the primary ADFS server
$AdfsSvcUsername is the username of the ADFS service account

Save the file with a name of your choice (do not close the file as we will be adding more lines to it). I named my script AddToADFSFarm.ps1

Next, we need to define a variable that will contain the path to the directory where the certificate file and the file containing the encrypted adfs service account password are stored. Also, we need a variable to contain the key that was used to encrypt the adfs service account password. This will be required to decrypt the password.

Add the following to the CSE file

$localpath = "C:\Program Files\WindowsPowerShell\Modules\Certificates\"
$Key = (3,4,2,3,56,34,254,222,1,1,2,23,42,54,33,233,1,34,2,7,6,5,35,43)

Next, we need to decrypt the encrypted adfs service account password.

Now, we need to import the certificate into the local computer certificate store. To make things simple, when the certificate was exported from the primary ADFS server, it was encrypted using the adfs service account password.

After importing the certificate, we will read it to get its thumbprint.

Up until now, the steps are very similar to creating an ADFS farm. However, below is where the steps diverge.

Add the following lines to add the virtual machine to the existing ADFS farm

You now have a custom script extension file that will add a virtual machine as a secondary node to an existing ADFS Farm.

Below is the full CSE

All that is missing now is the method to bootstrap the scripts described above using Azure Resource Manager templates.

Below is the ARM template that can be used to install ADFS role on the a virtual machine and then add this virtual machine as a secondary node to the ADFS farm

In the above ARM template, the parameter ADFS02VMName refers to the hostname of the virtual machine that will be added to the ADFS Farm.

Listed below are the variables that have been used in the ARM template above

The above method can be used to add as many nodes to the ADFS farm as needed.

I hope this comes in handy when creating an ARM template to automatically deploy an ADFS Farm with additional nodes.

Automate ADFS Farm Installation and Configuration

Introduction

In this multi-part blog, I will be showing how to automatically install and configure a new ADFS Farm. We will accomplish this using Azure Resource Manager templates, Desired State Configuration scripts and Custom Script Extensions.

Overview

We will use Azure Resource Manager to create a virtual machine that will become our first ADFS Server. We will then use a desired state configuration script to join the virtual machine to our Active Directory domain and then to install the ADFS role. Finally, we will use a Custom Script Extension to install our first ADFS Farm.

Install ADFS Role

We will be using the xActiveDirectory and xPendingReboot experimental DSC modules.

Download these from

https://gallery.technet.microsoft.com/scriptcenter/xActiveDirectory-f2d573f3

https://gallery.technet.microsoft.com/scriptcenter/xPendingReboot-PowerShell-b269f154

After downloading, unzip the file and  place the contents in the Powershell modules directory located at $env:ProgramFiles\WindowsPowerShell\Modules (unless you have changed your systemroot folder, this will be located at C:\ProgramFiles\WindowsPowerShell\Modules )

Open your Windows PowerShell ISE and lets create a DSC script that will join our virtual machine to the domain and also install the ADFS role.

Copy the following into a new Windows Powershell ISE file and save it as a filename of your choice (I saved mine as InstallADFS.ps1)

In the above, we are declaring some mandatory parameters and some variables that will be used within the script

$MachineName is the hostname of the virtual machine that will become the first ADFS server

$DomainName is the name of the domain where the virtual machine will be joined

$AdminCreds contains the username and password for an account that has permissions to join the virtual machine to the domain

$RetryCount and $RetryIntervalSec hold values that will be used to  check if the domain is available

We need to import the experimental DSC modules that we had downloaded. To do this, add the following lines to the DSC script

Import-DscResource -Module xActiveDirectory, xPendingReboot

Next, we need to convert the supplied $AdminCreds into a domain\username format. This is accomplished by the following lines (the converted value is held in $DomainCreds )

Next, we need to tell DSC that the command needs to be run on the local computer. This is done by the following line (localhost refers to the local computer)

Node localhost

We need to tell the LocalConfigurationManager that it should continue with the configuration after a reboot, reboot the server if needed and to just apply the settings only once (DSC can apply a setting and constantly monitor it to check that it has not been changed. If the setting is found to be changed, DSC can re-apply the setting. In our case we will not do this, we will apply the setting just once).

Next, we need to check if the Active Directory domain is ready. For this, we will use the xWaitForADDomain function from the xActiveDirectory experimental DSC module.

Once we know that the Active Directory domain is available, we can go ahead and join the virtual machine to the domain.

the JoinDomain function depends on xWaitForADDomain to finish successfully. If xWaitForADDomain fails, JoinDomain will not run

Once the virtual machine has been added to the domain, it needs to be restarted. We will use xPendingReboot function from the xPendingReboot experimental DSC module to accomplish this

Next, we will install the ADFS role on the virtual machine

Our script has now successfully added the virtual machine to the domain and installed the ADFS role on it. You now have to create a zip file with InstallADFS.ps1 and upload it to a location that Azure Resource Manager can access (I would recommend uploading to GitHub). Include the xActiveDirectory and xPendingReboot experimental DSC modules. Also add a folder called Certificates inside the zip file and put the ADFS certificate and the encrypted password files (discussed in the next section) inside the folder.

In the next section, we will configure the ADFS Farm.

The full InstallADFS.ps1 DSC script is pasted below

Create ADFS Farm

Once the ADFS role has been installed, we will use Custom Script Extensions (CSE) to create the ADFS farm.

One of the requirements to configure ADFS is a signed certificate. I used a 90 day trial certificate from Comodo.

There is a trick that I am using to make my certificate available on the virtual machine. If you bootstrap a DSC script to your virtual machine in an Azure Resource Manager template, the script along with all the non out-of-box DSC modules have to be packaged into a zip file and uploaded to a location that ARM can access. Then, before ARM uses the script, it downloads the zip file and unzips it, and copies the directories inside the zip file to $env:ProgramFiles\WindowsPowerShell\Modules ( C:\ProgramFiles\WindowsPowerShell\Modules )

I am using this feature to sneak my certificate on to the virtual machine. I create a folder called Certificates inside the zip file containing the DSC script and put the certificate inside it. Also, I am not too fond of passing plain passwords from my ARM template to the CSE, so I created two files, one to hold the encrypted password for the domain administrator account and the other to contain the encrypted password of the adfs service account. These two files are named adminpass.key and adfspass.key and will be placed in the Certificates folder within the zip file.

I used the following to generate the encrypted password files

$AdminSecurePassword = ConvertTo-SecureString {AdminPlainTextPassword} -AsPlainText -Force
ConvertFrom-SecureString $AdminSecurePassword -Key $key > adminpass.key

$ADFSSecurePassword = ConvertTo-SecureString {ADFSPlainTextPassword} -AsPlainText -Force
ConvertFrom-SecureString $ADFSSecurePassword -Key $key > adfspass.key

The $key  is used to convert the secure string into an encrypted standard string. Valid key lengths are 16, 24, 32

For this blog, we will use

$Key = (3,4,2,3,56,34,254,222,1,1,2,23,42,54,33,233,1,34,2,7,6,5,35,43)

Open Windows Powershell ISE and paste the following (save the file with a name of your choice. I saved mine as ConfigureADFS.ps1)

param (
 $DomainName,
 $DomainAdminUsername,
 $AdfsSvcUsername
)

These are the parameters that will be passed to the CSE

$DomainName is the name of the Active Directory domain
$DomainAdminUsername is the username of the domain administrator account
$AdfsSvcUsername is the username of the ADFS service account

Next, we will define the value of the Key that was used to encrypt the password and the location where the certificate and the encrypted password files will be placed

$localpath = "C:\Program Files\WindowsPowerShell\Modules\Certificates\"
$Key = (3,4,2,3,56,34,254,222,1,1,2,23,42,54,33,233,1,34,2,7,6,5,35,43)

Now, we have to read the encrypted passwords from the adminpass.key and adfspass.key file and then convert them into a domain\username format

Next, we will import the certificate into the local computer certificate store. We will mark the certificate exportable and set the password same as the domain administrator password.

In the above after the certificate is imported,  $cert is used to hold the certificate thumbprint

Next, we will configure the ADFS Farm

The ADFS Federation Service displayname is set to Active Directory Federation Service and the Federation Service Name is set to fs.adfsfarm.com

Upload the CSE to a location that Azure Resource Manager can access (I uploaded my script to GitHub)

The full ConfigureADFS.ps1 CSE is shown below

Azure Resource Manager Template Bootstrapping

Now that the DSC and CSE scripts have been created, we need to add them in our ARM template, straight after the virtual machine is provisioned.

To add the DSC script, create a DSC extension and link it to the DSC Package that was created to install ADFS. Below is an example of what can be used

The extension will run after the ADFS virtual machine has been successfully created (referred to as ADFS01VMName)

The MachineName, DomainName and domain administrator credentials are passed to the DSC extension.

Below are the variables that have been used in the json file for the DSC extension (I have listed my GitHub repository location)

Next, we have to create a Custom Script Extension to link to the CSE for configuring ADFS. Below is an example that can be used

The CSE depends on the ADFS virtual machine being successfully provisioned and the DSC extension that installs the ADFS role to have successfully completed.

The DomainName, Domain Administrator Username and the ADFS Service Username are passed to the CSE script

The following contains a list of the variables being used by the CSE (the example below shows my GitHub repository location)

"repoLocation": "https://raw.githubusercontent.com/nivleshc/arm/master/",
"ConfigureADFSScriptUrl": "[concat(parameters('repoLocation'),'ConfigureADFS.ps1')]",

That’s it Folks! You now have an ARM Template that can be used to automatically install the ADFS role and then configure a new ADFS Farm.

In the next blog, we will explore how to add another node to the ADFS Farm and we will also look at how we can automatically create a Web Application Proxy server for our ADFS Farm.

Create a new Active Directory Forest using Desired State Configuration

Desired State Configuration (DSC) is a declarative language in which you state “what” you want done instead of going into the nitty gritty level to describe exactly how to get it done. Jeffrey Snover (the inventor of PowerShell) quotes Jean-Luc Picard from Star Trek: The Next Generation to describe DSC – it tells the servers to “Make it so”.

In this blog, I will show you how to use DSC to create a brand new Active Directory Forest. In the next blog, for redundancy, we will add another domain controller to this new Active Directory Forest.

The DSC configuration script can be used in conjunction with Azure Resource Manager to deploy an new Active Directory Forest with a single click, how good is that!

A DSC Configuration script uses DSC Resources to describe what needs to be done. DSC Resources are made up of PowerShell script functions, which are used by the Local Configuration Manager to “Make it so”.

Windows PowerShell 4 and Windows PowerShell 5.0, by default come with a limited number of DSC Resources. Don’t let this trouble you, as you can always install more DSC modules to extend this library! It’s as easy as downloading them from a trusted location and placing them in the PowerShell modules directory! 

To get a list of DSC Resources currently installed, within PowerShell, execute the following command

Get-DscResource

Out of the box, there is no DSC modules available to create an Active Directory Forest. So, to start off, lets go download a module that will do this for us.

For our purpose, we will be installing the xActiveDirectory PowerShell module, which can be downloaded from https://gallery.technet.microsoft.com/scriptcenter/xActiveDirectory-f2d573f3  (updates to the xActiveDirectory module is no longer being provided at the above link, instead these are now published on GitHub at https://github.com/PowerShell/xActiveDirectory  . For simplicity, we will use the TechNet link above)

Download the following DSC modules as well

https://gallery.technet.microsoft.com/scriptcenter/xNetworking-Module-818b3583

https://gallery.technet.microsoft.com/scriptcenter/xPendingReboot-PowerShell-b269f154

After downloading the zip files, extract the contents and place them in the PowerShell modules directory located at $env:ProgramFiles\WindowsPowerShell\Modules folder

($env: is a PowerShell reference  to environmental variables).

For most systems, the modules folder is located at C:\ProgramFiles\WindowsPowerShell\Modules

However, if you are unsure, run the following PowerShell command to get the location of $env:ProgramFiles 

Get-ChildItem env:ProgramFiles

Tip: To get a list of all environmental variables, run Get-ChildItem env:

 

With the pre-requisites taken care of, let’s start creating the DSC configuration script.

Open your Windows PowerShell IDE and let’s get started.

Paste the following into your PowerShell editor and save it using a filename of your choice (I have saved mine as CreateNewADForest.ps1)

Configuration CreateNewADForest {
param(
      [Parameter(Mandatory)]
      [String]$DomainName,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$AdminCreds,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$SafeModeAdminCreds,

      [Parameter(Mandatory)]
      [System.Management.Automation.PSCredential]$myFirstUserCreds,

      [Int]$RetryCount=20,
      [Int]$RetryIntervalSec=30
)

The above declaration defines the parameters that will be passed to our  configuration function.  There are also two “constants” defined as well. I have named my configuration function CreateNewADForest which coincides with the name of the file it is saved in – CreateNewADForest.ps1

Here is some explanation regarding the above parameters and contants

$DomainName         -  FQDN for the Active Directory Domain to create
$AdminCreds         -  a PSCredentials object that contains username and password 
                       that will be assigned to the Domain Administrator account
$SafeModeAdminCreds -  a PSCredentials object that contains the password that will
                       be assigned to the Safe Mode Administrator account
$myFirstUserCreds   -  a PSCredentials object that contains the username and
                       password for the first domain user account to create
$RetryCount         -  defines how many retries should be performed while waiting
                       for the domain to be provisioned
$RetryIntervalSec   -  defines the seconds between each retry to check if the 
                       domain has been provisioned 

 

We now have to import the DSC modules that we had downloaded, so add the following line to the above Configuration Script

Import-DscResource -ModuleName xActiveDirectory, xNetworking, xPendingReboot

We now need to convert $AdminCreds the format domain\username. We will store the result in a new object called $DomainCreds 

[System.Management.Automation.PSCredential]$DomainCreds = New-Object System.Management.Automation.PSCredential ("${DomainName}\$($Admincreds.UserName)", $Admincreds.Password)

 

The next line states where the DSC commands will run. Since we are going to run this script from within the newly provisioned virtual machine, we will use localhost as the location

So enter the following

Node localhost
{

Next, we need to tell the Local Configuration Manager to apply the settings only once, reboot the server if needed (during our setup) and most importantly, to continue on with the configuration after reboot. This is done by using the following lines

 

LocalConfigurationManager
{
     ActionAfterReboot = 'ContinueConfiguration'
     ConfigurationMode = 'ApplyOnly'
     RebootNodeIfNeeded = $true
}

 

If you have worked with Active Directory before, you will be aware of the importance of DNS. Unfortunately, during my testing, I found that a DNS Server is not automatically deployed when creating a new Active Directory Forest. To ensure a DNS Server is present before we start creating a new Active Directory Forest, the following lines are needed in our script

WindowsFeature DNS
{
     Ensure = "Present"
     Name = "DNS"
}

The above is a declarative statement which nicely shows the “Make it so” nature of DSC. The above lines are asking the DSC Resource WindowsFeature to Ensure DNS (which refers to DNS Server) is Present. If it is not Present, then it will be installed, otherwise nothing will be done. This is the purpose of the Ensure = “Present” line. The word DNS after WindowsFeature is just a name I have given to this block of code.

Next, we will leverage on the DSC function xDNSServerAddress (from the xNetworking module we had installed) to set the local computer’s DNS Settings, to its loopback address. This will make the computer refers to the newly installed DNS Server.

xDnsServerAddress DnsServerAddress
{
     Address        = '127.0.0.1'
     InterfaceAlias = 'Ethernet'
     AddressFamily  = 'IPv4'
     DependsOn = "[WindowsFeature]DNS"
}

Notice the DependsOn = “[WindowsFeature]DNS” in the above code. It tells the function that this block of code depends on the [WindowsFeature]DNS section. To put it another way, the above code will only run after the DNS Server has been installed. There you go, we have specified our first dependency.

Next, we will install the Remote Server Administration Tools

WindowsFeature RSAT
{
     Ensure = "Present"
     Name = "RSAT"
}

Now, we will install the Active Directory Domain Services feature in Windows Server. Note, this is just installation of the feature. It does not create the Active Directory Forest.

WindowsFeature ADDSInstall
{
     Ensure = "Present"
     Name = "AD-Domain-Services"
}

 

So far so good. Now for the most important event of all, creating the new Active Directory Forest. For this we will use the xADDomain function from the xActiveDirectory module.

xADDomain FirstDC
{
     DomainName = $DomainName
     DomainAdministratorCredential = $DomainCreds
     SafemodeAdministratorPassword = $SafeModeAdminCreds
     DatabasePath = "C:\NTDS"
     LogPath = "C:\NTDS"
     SysvolPath = "C:\SYSVOL"
     DependsOn = "[WindowsFeature]ADDSInstall","[xDnsServerAddress]DnsServerAddress"
}

 

Notice above, we are pointing the DatabasePath, LogPath and SysvolPath all on to the C: drive. If you need these to be on a separate volume, then ensure an additional data disk is added to your virtual machine and then change the drive letter accordingly in the above code. This section has a dependency on the server’s DNS settings being changed.

If you have previously installed a new Active Directory forest, you will remember how long it takes for the configuration to finish. We have to cater for this in our DSC script, to wait for the Forest to be successfully provisioned, before proceeding. Here, we will leverage on the xWaitForADDomain DSC function (from the xActiveDirectoy module we had installed).

xWaitForADDomain DscForestWait
{
     DomainName = $DomainName
     DomainUserCredential = $DomainCreds
     RetryCount = $RetryCount
     RetryIntervalSec = $RetryIntervalSec
     DependsOn = "[xADDomain]FirstDC"
}

 

Viola! The new Active Directory Forest has been successfully provisioned! Let’s add a new user to Active Directory and then restart the server.

xADUser FirstUser
{
     DomainName = $DomainName
     DomainAdministratorCredential = $DomainCreds
     UserName = $myFirstUserCreds.Username
     Password = $myFirstUserCreds
     Ensure = "Present"
     DependsOn = "[xWaitForADDomain]DscForestWait"
}

The username and password are what had been supplied in the parameters to the configuration function.

That’s it! Our new Active Directory Forest is up and running. All that is needed now is a reboot of the server to complete the configuration!

To reboot the server, we will use the xPendingReboot module (from the xPendingReboot package that we installed)

xPendingReboot Reboot1
{
     Name = "RebootServer"
     DependsOn = "[xWaitForADDomain]DscForestWait"
}

 

Your completed Configuration Script should look like below (I have taken the liberty of closing off all brackets and parenthesis)

Configuration CreateNewADForest {
param
  (
     [Parameter(Mandatory)]
     [String]$DomainName,

     [Parameter(Mandatory)]
     [System.Management.Automation.PSCredential]$AdminCreds,

     [Parameter(Mandatory)]
     [System.Management.Automation.PSCredential]$SafeModeAdminCreds,

     [Parameter(Mandatory)]
     [System.Management.Automation.PSCredential]$myFirstUserCreds,

     [Int]$RetryCount=20,
     [Int]$RetryIntervalSec=30
  )
  Import-DscResource -ModuleName xActiveDirectory, xNetworking, xPendingReboot
  [System.Management.Automation.PSCredential]$DomainCreds = New-Object System.Management.Automation.PSCredential ("${DomainName}\$($Admincreds.UserName)", $Admincreds.Password)

  Node localhost
  {
     LocalConfigurationManager
     {
         ActionAfterReboot = 'ContinueConfiguration'
         ConfigurationMode = 'ApplyOnly'
         RebootNodeIfNeeded = $true
     }

     WindowsFeature DNS
     {
         Ensure = "Present"
         Name = "DNS"
     }

     xDnsServerAddress DnsServerAddress
     {
          Address        = '127.0.0.1'
          InterfaceAlias = 'Ethernet'
          AddressFamily  = 'IPv4'
          DependsOn = "[WindowsFeature]DNS"
     }

     WindowsFeature RSAT
     {
          Ensure = "Present"
          Name = "RSAT"
     }

     WindowsFeature ADDSInstall
     {
          Ensure = "Present"
          Name = "AD-Domain-Services"
     }

     xADDomain FirstDC
     {
          DomainName = $DomainName
          DomainAdministratorCredential = $DomainCreds
          SafemodeAdministratorPassword = $SafeModeAdminCreds
          DatabasePath = "C:\NTDS"
          LogPath = "C:\NTDS"
          SysvolPath = "C:\SYSVOL"
          DependsOn = "[WindowsFeature]ADDSInstall","[xDnsServerAddress]DnsServerAddress"
     }

     xWaitForADDomain DscForestWait
     {
          DomainName = $DomainName
          DomainUserCredential = $DomainCreds
          RetryCount = $RetryCount
          RetryIntervalSec = $RetryIntervalSec
          DependsOn = "[xADDomain]FirstDC"
     }

     xADUser FirstUser
     {
          DomainName = $DomainName
          DomainAdministratorCredential = $DomainCreds
          UserName = $myFirstUserCreds.Username
          Password = $myFirstUserCreds
          Ensure = "Present"
          DependsOn = "[xWaitForADDomain]DscForestWait"
     }

     xPendingReboot Reboot1 
     { 
          Name = "RebootServer" 
          DependsOn = "[xWaitForADDomain]DscForestWait"
     }
  }
}

 

You can copy the above script to a newly create Azure virtual machine and deploy it manually.

However, a more elegant method will be to bootstrap it to your Azure Resource Manager virtual machine template.

One caveat is, the DSC configuration script has to be packaged into a zip file and then uploaded to a location that Azure Resource Manager can access (that means it can’t live on your local computer).

You can upload it to your Azure Storage blob container and use a shared access token to give access to your script or upload it to a public repository like GitHub.

I have packaged and uploaded the script to my GitHub repository at

https://raw.githubusercontent.com/nivleshc/arm/master/CreateNewADForest.zip

The zip file also contains the additional DSC Modules that were downloaded.

To use the above in your Azure Resource Manager template, create a PowerShell DSC Extension and link it to the Azure virtual machine that will become your Active Directory Domain Controller (referred to as DC01 in the code below).

"resources": [
  {
    "type": "Microsoft.Compute/virtualMachines/extensions",
    "name": "[concat(parameters('DC01Name'),'/CreateNewADForest')]",
    "location": "[resourceGroup().location]",
    "apiVersion": "2015-06-15",
    "dependsOn": [
         "[concat('Microsoft.Compute/virtualMachines/', parameters('DC01Name'))]"
    ],
    "tags": {
         "displayName": "CreateNewADForest"
    },
    "properties": {
         "publisher": "Microsoft.Powershell",
         "type": "DSC",
         "typeHandlerVersion": "2.19",
         "autoUpgradeMinorVersion": true,
         "settings": {
              "Modulesurl": "[variables('CreateNewADForestPackageURL')]",
              "ConfigurationFunction":"[variables('CreateNewADForestConfigurationFunction')]",
              "Properties": {
                   "DomainName": "[parameters('domainName')]",
                   "AdminCreds": {
                        "UserName": "[parameters('adminUserName')]",
                        "Password": "PrivateSettingsRef:AdminPassword"
                   },
                   "SafeModeAdminCreds": {
                        "UserName": "[parameters('safemodeAdminUserName')]",
                        "Password": "PrivateSettingsRef:SafeModeAdminPassword"
                   },
                   "myFirstUserCreds": {
                        "UserName": "[parameters('myFirstUserName')]",
                        "Password": "PrivateSettingsRef:MyFirstUserPassword"
                   }

              }
         },
         "protectedSettings": {
               "Items": {
               "AdminPassword": "[parameters('adminPassword')]",
               "SafeModeAdminPassword": "[parameters('safemodeadminPassword')]",
               "MyFirstUserPassword": "[parameters('myFirstUserPassword')]"
         }
    }
  }
]

 

Below is an except of the relevant variables

"repoLocation": "https://raw.githubusercontent.com/nivleshc/arm/master/",
"CreateNewADForestPackageURL": "[concat(variables('repoLocation'), 'CreateNewADForest.zip')]",
"CreateNewADForestConfigurationFunction": "CreateNewADForest.ps1\\CreateNewADForest",

And these are the relevant parameters

That’s it folks! Now you have a new shiny Active Directory Forest, that was created for you using a DSC configuration script.

In the second part of this two-part series, we will add an additional domain controller to this Active Directory Forest using DSC configuration script.

Let me know what you think of the above information.

Read More »