Using Slack for Logging (with ServiceStack) – Part 1

This post is about using Slack (“team communication for the 21st century team”) as a logging tool with ServiceStack. Rather than using the conventional approach of logging to a text file or (even worse) the Windows Events approach, I will use Slack for logging such that Log.Info(“This is magic”) will post a Slack message to the relevant Slack channel & using Slack’s awesome tooling, all participants are immediately notified.

loggerInfoToSlack.png

What is Slack?

If you are reading this & you work in a team then you should be using Slack. Period. For more convincing arguments, see the Quora post here on What makes Slack a must use Product. The important points are, tt’s free and will revolutionize the way you interact with your colleagues. What really differentiates this product from others (such as Skype), is the ease of integrating third party services (like GitHub, Asana, Google and/or custom integrations (as we will see in this post)) to post messages directly to Slack channels. This opens the door in so many ways different ways .

Slack as a Logging Tool!?! Is it not a chat app?!

So is it weird for Slack to be used as a Logging tool?! Not really, if you consider the most widely used practice for logging is to a random text file somewhere on the server. In reality this means getting a call from your boss (or you making a call, gasp!) at 1am, spending valuable sleep time getting onto the server, finding the error log & then realising you have load-balancing in place, which means it is on another server (this is something that really happened! Argh no jokes). No thanks, I choose the Slack app for firstly [turning message notificatons on] for the really, really important stuff  (like high level service exceptions) and [turning off] notifications for any other stuff (I value my sleep, you know).

Ok, I am convinced. How do I implement Slack with ServiceStack?

ServiceStack provides an implementation-free ILog and ILogFactory interface that our application logic can bind to. This allows us to create our own custom logger (the SlackLogger) easily. Thus when we call Log.Info(“Magic was used to post this message to Slack”), it is sent to our Team’s slack channel and all channel participants are notified immediately.

A. Setting up Slack

We will need to set up Slack in order to get our custom webhook url, which is the ‘key’ to post Slack notifications.

  1. Go to https://api.slack.com/ and Register as a developer (believe me, this is very very quick!)
  2. Then click on ‘Start building custom integrations’

Screen Shot 2016-02-01 at 7.45.12 AM.png

3. Click on [Set up incoming webhooks]

Screen Shot 2016-02-01 at 7.47.55 AM

4. Sign into your relevant team & either select an existing / default channel you would like to post to or create a new channel. [Note even though this is the default channel, you are able to send messages to other channels using the same webhook]

Screen Shot 2016-02-01 at 7.49.36 AM.png

 

5. Copy the Webhook URL

Screen Shot 2016-02-01 at 7.51.53 AM.png

Posting to Slack with our custom SlackLogger

We will need to create a class ‘SlackLogger’ that inherits from ILog.
Then, we only need to remember two important points.
Firstly, the  incomingWebHookUrl (this is the Slack ‘key’ we got above).
Secondly, the line incomingWebHookUrl.PostToJsonUrl( new { text = message.ToString()}); 

incomingWebHookUrl.PostToJsonUrl( new { text = “random message });
I love simplicity of this one line allowed by ServiceStack’s HttpUtils. Especially when we compare this to the .NET approach of creating proxies and abstraction classes to wrap the WebRequest (ugly).
With this in place & making some messages more informative (e.g preceding info logs with “Info:”, we can now register this new SlackLogger in our app.
Registering SlackLogger in our App using IoC

Using it in your app:
Given that we have registered in our AppHost class, we are now able to use IoC in the services that need it, by simply instantiating it with public ILog Logger {get;set;} and then calling the relevant method such as Log.Info(“This is magic”) which will send a message to Slack.
Screen Shot 2016-02-01 at 8.46.08 AM.png

Summary
We have shown above how to set up Slack for a custom integration.
We have implemented our own SlackLog (ILog) implementation & registered it in our Application. Using Funq as the IoC container, we are able to call it in any service & send Slack notification messages.

ElasticSearch and ServiceStack

In this post,  I will show you how to integrate Full text searching using ElasticSearch and ServiceStack.

Options for Full Text Search

Before I dig into the steps required for ElasticSearch, I will mention some other options for implementing full-text in your solution:

  • Sphinx  – used by craigslist
  • Solr – used by CNet, Netflix, digg.com
  • ElasticSearch – used by Foursquare, Github
    (source: StackOverflow, thanks Tommy for highlighting use cases)
  • given that I use PostgreSQL, I could also just use FTS in PostgreSQL 

In the .NET. world, the above solutions have got .NET connectors. For

  • SpinxConnector.NET  – very good! Unfortunately it costs EUR199 for a single dev license. If it were free, I probably would have used this one.
  • SolrNET – is an active GitHub project with quite a few contributors. If you are looking to implement Solr in .NET, this is probably the way to go.
  • NEST  for ElasticSearch – this is the official high level ElasticSearch client for .NET. I chose to go with this option & there are some code examples below on how I implemented it.

Installing ElasticSearch in Windows

Summary:
1st step – install Java
& set the JAVA_HOME variable
2nd step – download & install ElasticSearch
3rd step – install NEST
as a NugetPackage in VisualStudio

Details on above:

1st step – install Java

a. Go to Java.com.

b. Click on the Free Java Download button, and then tap or click Agree and Start Free Download & install

c. in order for ElasticSearch to run, we will need to set the JAVA_HOME variable. Otherwise you will get an error ‘no JAVA_HOME variable set’

For setting the JAVA_HOME in Windows,

  • click on Start Button & Configure Java
  • Navigate to the [JAVA] Tab  & click on [View…]
  • Copy the Path (as per below screenshot) without the \bin\javaw.exeScreen Shot 2016-01-19 at 7.50.12 PM
  • In Windows 7,
    – Right-click the My Computer icon on your desktop and select Properties.
    – Click the Advanced tab, then click the Environment Variables button.
    – Under System Variables, click New.
    – Enter the variable name as JAVA_HOME.
    – Enter the variable value as the installation path as per above
    – Apply changes

Screen Shot 2016-01-21 at 7.30.12 AM.png

2nd step – download & install ElasticSearch

a. Download ElasticSearch in zip format from the official ElasticSearch homepage & unzip into a specified directory. (For me this is , C:\Projects\ElasticSearch)

b. To run ElasticSearch in a console app,
– go to directory (cd C:\Projects\ElasticSearch\bin)
– & run ElasticSearch in console (elasticsearch.bat)

Since ElasticSearch is itself a REST webservice (which by default runs on port 9200), you can now open a browser and go to localhost:9200 & be presented with something similar as per below.

Screen Shot 2016-01-21 at 7.33.25 AM.png

In the long run, you would want to run this as a Windows service. However, whilst we are developing, I will run this as a console app.

3rd step – install NEST in Visual Studio

ElasticSearch is a RESTful web service which exposes great JSON output. To make interacting with this service easier, I will work with NEST. This is the official high-level .NET client which can map our requests and responses to strongly typed objects. It comes with a powerful DSL which maps one-to-one with ElasticSearch. I will highlight a few code samples in the ‘Show me the Code’ sections.

Adding NEST to our project could not be easier. We can simply install it using Nuget package manager.

PM > Install-Package NEST

 

Show me the Code – Indexing Data

The above code,

  • firstly shows how we can take a Request Dto and using ServiceStack’s auto-mapping feature convert the requestDto to our Listing object.  (This is similar to Jimmy Bogard‘s widely used AutoMapper)
  • secondly, we persist our Listing object to our own database (in this case I am using PostgreSQL). [Again, ServiceStack’s ORMLite functions out-of-the-box with almost any well-known backend RDBS database, including SQL, MySQL,  PostgreSQL, Firebird, Oracle)
  • and finally, we get to the indexing part:
    – firstly we get a connection to the Elastic webservice
    – & then call .Index to index the object.
    Now we let ElasticSearch webservice do the magic. Indexing is as easy as that!

 

Show me the Code – Searching Data

I have to be honest, the ElasticSearch search syntax was not the easiest for me to understand. I found it best working with Sense (see screenshot below). Sense is a chrome extension (you can get it on the Google App store for free, which allows you to construct Elastic search terms & query your ElasticSearch webservice in a more raw format). Since NEST maps to the ElasticSearch DSL one-to-one, it is easier first to figure out & play around with constructing queries in this format. Once you are happy with what you want to achive, you can then simply construct them in NEST.

Screen Shot 2016-01-22 at 7.33.15 AM.png

Constructing Search requests in NEST & querying ElasticSearch

In the above code, I have

  • my ElasticSearchClient. This is injected at run-time using ServiceStack’s IOC container. (see code & how it is done by going to this stackoverflow post. [Note, I had some initial issues with this. So the post shows me asking the question & Demis (lead developer) answering that question within a day! (this is typical of SS support)]
  • I then show all the request parameters, which are part of the ‘GetListings’ request Dto. You will note that for most of these, if they are null, I am using the wildcard operator (*) to match to anything.
  • Thirdly, I start constructing the Search request. For me, it was easiest for breaking it out into separate parts & then constructing the SearchRequest in the end. (I feel it gives the best oversight for me).
  • Finally, I call Elastic.Search(searchRequest). You will note that pageNumber, pageSize, I can enter in the searchRequest. Also #totalRecords are returned from the ElasticSearch response object. Beautiful, this makes my paging functionality very easy!

 

Summary

In this post I managed to implement quick, relevant searches for end-users coming to search my Classifieds site (see reasons of building a Classifieds site here & my income reports of the site here) using ElasticSearch & integrating with the ServiceStack web services framework. ElasticSearch is easily extendable. Thus for now it may be overkill, but I know it will meet my future needs & was far easier to set up and go than I had anticipated. Kudos to the ElasticSearch team & ServiceStack for providing awesome frameworks.

Finally, for those looking to get future updates of my story of building this Classifieds site & sharing my monetization strategy (yip, I am hoping to make some money off the Classifieds site); then sign up here for my mailing list (takes you to my mailchimp link. Promise I will not ever spam!)

Adding Facebook Authentication using ServiceStack

I am building a Classifieds site and will be sharing income reports and stats once launched  (click here to sign up to my mailing list)

In this post, I will be sharing how I added Facebook authentication to my website using ServiceStack. One users are logged in, they are able to post classified ads.

loginFaceBook

How to add Facebook authentication using ServiceStack

Facebook offers OAuth integration for developers to allow users to log in to external applications using their Facebook account. ServiceStack has a done the hard work for us already for Facebook auth integration (others include Twitter, Google) & all we have to do is plug it into our app. These are the steps:

Steps:

Registering with Facebook

  1. You will need to register with Facebook as a developer
    – go to https://developers.facebook.com & click on register
  2. Click on Add a new App & in my case I will add a new WebsitefaceBookAddANewApp.png
  3. Since my Site is not launched yet, I elect the [create new AppId] & choose a unique name for it.
  4. Take note of the AppId and AppSecret,which we will need for our configuration Files
    & (importantly) you will also need to add your Website URL by clicking on [Add Platform]. (Otherwise you receive an ugly error)
    faceBookAppIds.png

Configuring Facebook Authentication in our ServiceStack app

  1. We will need to add the AuthFeature plugin. This allows multiple auth providers (Twitters). We will add FacebookAuthProvider to this list.
  2. You can provide the AppId, AppSecret (as provided by Facebook) in code (as per below) or via Web.Config
  3. Redirect, Callback Urls need to be set. These are used by ServiceStack’s OAuth & include a callback URL as a query string to the Facebook.
  4. Set permissions. By default you will have access to the user’s public profile, list of friends & email. You can set additional permissions (note these have to be approved first by Facebook) using Permissions. [For a full list of Facebook permissions go to https://developers.facebook.com/docs/facebook-login/permissions]

In our website, we will create a login button, which is shown when the user is not authenticated. If authenticated, we will show the user’s default profile.

Login Button shown if not authenticated

Screen Shot 2016-01-14 at 6.57.55 AM

Profile Image & Logout shown if authenticated

Screen Shot 2016-01-14 at 6.57.18 AM

 

 

Finally, lets require [Authenticate]ion for specific user actions

In my Classifieds website, I only want to allow authenticated users to post Classified ads. This is easily done with ServiceStack. I can just decorate the necessary method or Dto with [Authenticate].

Digging into some data: behind the scenes of AuthFeature,

AuthFeature uses IUserAuthRepository to store user authentication details. We can use the standard OrmLiteAuthUserRepository class (as in the above code). This will create the tables as per below [I am using PostreSQL as the backend. Again this works seamlessly with ServiceStack’s ORMLite].

OrmAuthTables

ormAuthTables.png

Pagination using ServiceStack, BootStrap & jQuery

In this post, I’ll explain how I introduced paging into my Classifieds project. There are a number of different ways to accomplish this, including writing your own jQuery plugin.

However, where possible, I prefer to take some shortcuts as I am in need of a quick solution & there are some great / well tested plugins out there. So for my Bootstrap classifieds site, I’ll be using

  • ServiceStack – (this is my go-to webservice framework for this entire Classifieds project). This will return the resulting list of records. Importantly, it also includes total page count, current page number which we can make use of on the client side.
  • jQuery Pagination plugin (optimised for Bootstrap) – is a well-written jquery pluging which I will use for the client-side pagination [ link to GitHub project]

paging

API – Reqest and Responses for Paging queries

The first thing I’ll do is write the necessary request (namely, GetListings) and response (GetListingsResponse) objects.

Thus an example request would use the api route ~/listings?CategoryId=1&PageNr=1, which returns the first page of listings within categoryId =1. This happens to be a Category of ‘Houses for Sale’ in my case]

Messages – Request Parameters as properties
Warning, here comes a tangent (skip this passage if you are only interested in the paging solution).

ServiceStack promotes a message based approach. In my case, I know there will be additional future request filters/parameters I will want to add such as (e.g. CarMake). Using this message based design, these additional request parameters can simply be added as properties. This promotes a very extendable & clean code base without breaking any existing code.

Compare this to to the more typical approach of having to amend the signature of a method (a typical method with request parameters is given below). Adding new parameters can get annoying quickly in these cases, as it requires modification of the existing method signature! (for more on this topic within an MVC project, see Jimmy Bogard’s post)

hmmmmphh – much uglier.

jQuery Pagination Plugin

Installing the plugin:
You will need to download the jquery file from their homepage & then reference it in your page.

http://~/scripts/jquery.twbsPagination.min.js

Configuring the plugin:

From the plugin’s homepage we can see that we can configure the plugin for a number of things. For my purposes, I will be using the response of the GetListings request & make use of the TotalPageCount & CurrentPageNr to pass into the jquery options function.

$(‘#pagination-control’).twbsPagination({
totalPages: @Model.TotalPageCount,
visiblePages: 5,
startPage: @Model.CurrentPageNr,
href: ‘?’ + params + ‘&PageNr={{number}}’
});

 

Setting the Page url using href propery

I will use the href option to pass the page url (corresponding to my listings route). I want to pass all the page’s current request parameters (e.g. the CategoryId) as well as pass the newly selected PageNr to the route. Not being a Javascript guy (!!), I ended up with a bit of an ugly implemetion for this. [If anyone has some cleaner code or better suggestion, please leave them in the comments]

Uploading Images – ServiceStack and Dropzone

I am building a Classifieds site (see list of reasons here).

In this post, I will be showcasing how I used DropZone (an awesome open-source library for drag-n-drop functionality) and ServiceStack (as our back-end service for uploading, resizing & storing the files) in providing users with a drag and drop facility for uploading their images. Once images are dropped, the service handles the resizing of images & saving of images onto the server.

uploadingImages_3

Installing & configuring DropZone
Installing dropzone
– You can install it using Nuget. Install-Package dropzone.
– Alternatively, you can download the latest release packages here.

The most important file is the dropzone.js. You will need to reference this in your code if you want to make use of it as per below. I suggest also referencing dropzone.css to make use of Dropzone’s awesome styling.

Dropzone configuration options
There are multiple configuration options available & DropZone makes it really easy.
For the Classifieds site, I wanted
– a maximum file size of 3Mb
– 10 files maximum per post,
– limiting parallel uploads to 1. This  prevents memory stream being accessed by another image before the image is finished [if there is another way of handling this, please provide feedback in the comments]

Finally,
– on successfully uploading, I want to pass the saved file location’s url to a hidden input field (“Files”).  Thus when a user posts the form, I can link the image URLs to the classified ad.

Again, Dropzone makes this very easy by listening to events. In this case the ‘success’ event.


Using ServiceStack for uploading, resizing & storing images
Dropzone requires a server-side implementation to upload, resize & restore images. I will be using ServiceStack for this.

Firstly, you notice that the dropzone element includes an action parameter.  “/images/upload/” This is the Servicestack api route which will handle the upload.

Now, the server-side code for ServiceStack. We first need to create the Request and Response DTOs . For those familiar with WFC or WebAPI, note how much simpler this is (a full description on differences is given in this stackoverflow post [new window]). The response DTO includes a simple URL which is the image’s saved location. (you can access response properties from DropZone – see DropZone on success script above)

The upload, resizing and storing is all housed in the ImageService (this code has been adopted and marginally changed from http://imgur.servicestack.net/. Thanks Demis). Specifically,
– I wanted uploaded files to receive a unique name. Thus brought in the RandomString(int length) method when creating a unique file name. Thanks dtb @stackoverflow.
– my ResponseDTO includes a URL (giving the image’s filename), which can be used by DropZone

 

Coming up in future posts:

  • I will make a decision between deploying to Azure or AWS
  • I will launch the actual Classifieds Site and use Facebook Ads & share the results
  • More cool ServiceStack features

Get notified