SaaS deployments are now ‘mission critical’

December 11, 2014/0 Comments/in Analytics, Blog, Technology /by Ash Nelson

Gartner recently published a survey citing that SaaS deployments are now ‘mission critical.’ Some of the key reasons behind this statement is that respondents looked for cost savings, an increase in innovation and accessibility to their systems as key drivers for the move away from local software solutions.

Joanne Correia, Gartner Research Vice President said,

“The most commonly cited reasons the survey found for deploying SaaS were for development and testing production/mission-critical workloads,” and went on to say “This is an affirmation that more businesses are comfortable with cloud deployments beyond the front office running salesforce automation (SFA) and email.”

This shows that companies are becoming more aware, and switched on to the benefits that cloud based software can bring to their company.

It was also demonstrated that on top of cost savings, accessibility, and innovation, SaaS based systems allowed for easier training and lower learning curves for employees.

“Non-IT professionals, often view the cloud strictly as a tool that they can use to reduce their operating costs,” and in turn effort.

Biarri empowering you through the cloud

Biarri was established in 2009 with the mission to provide accessible business optimisation to all clients regardless of size or budgets. We develop bespoke SaaS based solutions for you, with you allowing your solutions to meet your specific requirements.

We have been able to develop a bunch of applications for our clients to suit their specific; Advanced Planning and Scheduling, Workforce Management, Business Analytics and Supply Chain needs.

Get in touch and see how you can benefit from our solutions today!

Dynamic scheduling in the face of uncertainty

June 4, 2014/1 Comment/in Blog, Mathematics, Operations Research, Rail /by Dave Scerri

Many businesses operate in uncertain environments, at the mercy of breakdowns, human error, traffic or the weather. The traditional approach is to build slack into a schedule, to survive some expected level of uncertainty, and then to hope this schedule can be executed in spite of whatever unforeseen events actually occur. In instances where the schedule cannot be implemented as expected, operational decisions are made to try to meet the business goals as much as possible. Under complexity and time pressure, decision makers must often resort to making short term, locally focused decisions without the time or tools to consider implications downstream.

In this blog post, optimisation consultant Dave Scerri describes how recent algorithmic advances and smart tools have enabled the best of both worlds: an operational scheduling approach that responds to changing circumstances while focusing on system wide efficiency.

Dealing with operational uncertainty is one of the biggest challenges facing many businesses today. The most well thought out plans and schedules rarely survive contact with the real world, and the consequences are often a loss of productivity and overall efficiency. Businesses should be endeavouring to make the best decision at any point in time given the current circumstances and the best information available to them. Operational scheduling, when done right, can allow businesses to make quality, timely decisions, maximising their productivity in the face of uncertainty.

The best laid plans of mice and men often go awry

– Robert Burns

COTS vs Bespoke

May 14, 2014/0 Comments/in Blog, Business /by Ash Nelson

COTS (commercial off the shelf) puts the bars around accessible Mathematics: leads to crying babies

Our philosophy is to make the power of mathematics accessible. Why? Because we think it isn’t currently very accessible, this limits the number of people who can use it to get value and reduces the value derived by those who do use it, and that’s a crying shame in a world that desperately needs efficiency.

We have all seen it multiple times in multiple organisations. It’s the hard to use (probably ugly), not really fit for purpose (lots of workarounds), complicated IT (n tier, client/server, VM, Citrix, Oracle thing) approach to providing optimisation software.

So how did it come into being? Here’s how I see it:

“I’m unique; give me your shrink wrapped product!” – and other amusing procurement stories

Let’s assume requirements are done, I’ll save organisational scope bloat for another time. The next question is build or buy? How will we best get something that is a close match to need/requirements?

So a market search ensues only to discover that the requirements are pretty unique. So a custom/bespoke solution is required! That makes sense but most organisations quickly discover that bespoke = expensive (time and money), just like buying a tailor made suit is more expensive than buying off the rack.

It’s for this reason that hard core mathematics/optimisation solutions have mainly been consumed by capital intensive industries where spending a few million to save tens or hundreds of millions made the business case stack up.

Therefore organisations often seek a COTS (Commercial Off The Shelf) solution (often after an expensive run in with a bespoke approach), with the expectation that if they specify what they need and buy something “off the shelf” that fits then it should be low risk (time and money). It appears to be quite an entrenched view with Australian CIOs, and in some cases is justified, particularly in back office functions that don’t offer opportunity for differentiation. A point Wesfarmers Insurance CIO David Hackshall and DoD CIO Peter Lawrence make in an article by Brian Corrigan on itnews.com.au titled “How COTS became Australia’s default software setting”.

In the world of mathematics, optimisation and advanced planning and scheduling it would be a very rare occasion with a simple set of generic requirements where COTS really worked. Take one of the classical problems where mathematics are applied, vehicle routing. This is a well picked over area and sounds simple enough. Nonetheless, vendors fill niches within this niche in order to provide a match to requirements. As the Vehicle Routing survey in February 2014 issue OR/MS Today says “VR customers are different, and so are their routing needs and problems, which require flexible, innovative answers”.

Vendors react to this COTS centric procurement environment in a predictable way, and of course say they sell COTS because otherwise when they get evaluated on the inevitable RFX criteria they would fail miserably. The solution? They will (and I’ve been there) include “configuration” or “installation services” as ways to mask software development. The result? You get something that wasn’t a great fit with lots of add on development to meet your requirements. It’s hard to use, slow and doesn’t really provide the solutions you were hoping for. In many cases you end up with the worst of both worlds, the cost of bespoke but the poor fit of COTS.

As the aforementioned itnews.com.au article says “The middle ground between buying readymade software and building bespoke solutions is to customise a COTS package. Yet as many CIOs have discovered at great cost to their budgets and mental health, this can be a painful experience.”

This COTS/bespoke paradox is the problem we saw and it is what we aim to address. So what does Biarri do differently? We take the benefits of bespoke and make it cheaply and quickly. You could say we aim to provide the best of both worlds.

Do the math

How do we do it? First of all, we do the maths first! Prove you can solve the underlying problem and that’s it is worth solving before investing in the delivery mechanism. Once you know there is value in the maths, make sure people can digest it via a well-designed solution. The Biarri Workbench is our SaaS platform that allows us to very quickly develop easy to use, custom applications with unique workflows with an iterative/agile and light deployment.

Who says B2C owns good UX?

Easy to use means designed with the user in mind. In the consumer world (B2C) this is the natural order of things (thanks Apple). In the business world (B2B) this has taken a back seat, and that’s where our industrial designers come in. Working with users to really understand how they do their job and will interact with the system. Producing mock-ups/concepts and getting early feedback before a line of code is written.

So now we’ve proven the maths will provide value and designed a solution that users will love to use.

Mock up example

Rinse and Repeat

What comes next is turning this into reality quickly, cheaply and iteratively. Quickly and cheaply are thanks to the Biarri Workbench providing security, common database, existing UI components, libraries and widgets that enable a custom built application to be constructed very quickly. And “iteratively” is thanks to being web delivered which means we can provide early access to users to start providing feedback. Agile development takes on real meaning as users see the mock-ups they helped design come alive in their web browser mere weeks (or even just days) after designing them. Engagement and user buy-in are huge as feedback is provided, incorporated and delivered instantly. Australia Posts CIO Andrew Walduck understands this approach, “The number of times I’ve seen operating models where you start with requirements on one side, you dump it into operations on the other, and it fundamentally misses the point”.

Tool UI Example

It takes different strokes to move the world… yes it does

Do you remember the late 70’s early 80’s TV series “Different Strokes”? I use to love the theme song.

Everybody’s got a special kind of story
Everybody finds a way to shine,
It don’t matter that you got not alot
So what,
They’ll have theirs, and you’ll have yours, and I’ll have mine.
And together we’ll be fine….

When you start looking for your next optimisation, analytics or advanced planning and scheduling solution and your CIO/CFO says “budgets are tight and you can’t buy bespoke, you have to go COTS”, remember “it don’t matter that you got not a lot… you’ll have yours” because Biarri has a special kind of story.

How much tax is paid on your landfill?

April 30, 2013/0 Comments/in Blog, Business /by Ash Nelson

What do you know about Landfill Gas Emissions? I’m betting not much if you’re not directly working in the industry. As you’ve stared forlornly at the departing garbage truck at 5am after being awoken by its imminent arrival only to recall that you didn’t put the bins out last night, you probably haven’t given too much thought to where it’s going and what happens to the garbage.

Well let me ease your mind and put you off your breakfast. After leaving you in its dust (and your pyjamas), that truck was heading to one of about 450 landfills all around Australia. When it dumps its load of garbage, bacteria are going to start a feast on your leftover spaghetti bolognese and while doing so will emit gas. That gas they emit while eating your garbage is about half methane (CH4) and half carbon dioxide (C02), both of which are greenhouse gasses.

Heard of the carbon tax? Well in basic terms the Clean Energy Regulator, part of the Department of Climate Change and Energy Efficiency, maintains a database of entities expected to pay a carbon price. Landfills will have to pay a carbon price on new waste deposits from 1 July 2012 to encourage them to capture their methane emissions, which can then be used to produce heat or electricity. If a Landfill produces more than 25,000 tonnes of C02-equivalents a year they must report their emissions under the National Greenhouse and Energy Reporting (NGER) Scheme by 31 October 2013.

So if you operate a landfill how on earth do you calculate your emissions? Well it’s pretty tricky business and the National Greenhouse and Energy Reporting (Measurement) Determination 2008 provides methods and criteria for calculating greenhouse gas emissions and energy data under the National Greenhouse and Energy Reporting Act 2007 (NGER Act). Check out some of it here if you have trouble sleeping.

The government doesn’t expect you to know all that detail however and handily provides a calculator that you can use, which you can download on this page. It does the job but is pretty basic and doesn’t do things like show you the waste generated by each deposit in isolation of the totals. It also doesn’t give you anyway to visualise your data or change the parameters used in the calculation, which I’d be pretty interested in given the impending election in September.

The breakdown by deposit and visualisation is why Boral reached out to us for a better calculator last year to calculate emissions on the Boral Western Landfill, located west of Melbourne. So we made one for them. You can read about that here.

We only recently started to ask ourselves how other landfill operators were handling the same problem and whether they also wanted the greater capability of the Biarri Landfill Gas Emissions Calculator. So we did some research and reached out to some people we thought would know better than us about the industry. We were surprised to hear that the government tool was the only option.

So we looked into their tool further and found that they give users a handy guide, the “Solid Waste Disposal On Land User Guide V2.2”. Just don’t use it for calculating your emissions this year! As they say on page 6 “this estimate is not based on the NGER (Measurement) Determination for 2012/13. For 2012/13 reporting you will need to download an updated version of the Calculator when it is available”. And I’m sure it will be made available at some point but it isn’t available yet!

So we decided to make our tool more broadly available (yes, it is compliant with NGER (Measurement) Amendment Determination 2012/13 for the upcoming reporting period).

We’ve set up a site at http://landfill-emissions.biarri.com/ where you can download a trial of the tool and of course also buy it.

So now you know what happens to your rubbish and, if you’re a landfill operator, how to calculate your emissions this year. And by the way, it’s bin night!

Biarri Landfill Gas Emissions Calculator Screen Shots

Non-Legacy Quantity Generated

Annual Generation

Annual Non-Legacy Generation

Annual Total Generation

Forecast Landfill Qty (input)

Pre-emptive optimisation

September 28, 2011/0 Comments/in Blog, Operations Research, Programming /by Ash Nelson

For one of our long-standing clients we have been running vehicle routing optimisations on a daily basis. A file of daily orders is uploaded into our Workbench system, and is split up into several regions, each of which needs to be separately optimised. A planner works through each region, going through a series of data checks (e.g. location geocode checking), before hitting an “Optimise” button.

All of the heavy lifting is done on a server (the planner accesses it through a web app via a browser), so it’s possible that the server could silently start up the required optimisations without the planner’s involvement, and in the (fairly common) case where the region does not require any data corrections, when the planner is up to the optimisation stage, the result could be immediately available (as it has already been run, or is in the process of being run). This idea now been implemented and took only a short amount of Python code.

Furthermore, it runs in parallel as each optimisation is itself split into a separate child process (running a C++ exe) which Linux distributes across the 8 cores of our server machine.
The pre-emptive optimisations are kicked off using Python’s multiprocessing package, as follows:

from multiprocessing import Process

…

p = Process(target=start_preemptive_optimisations, args=(…))

p.start()

Control is returned to the user at this point while the optimisations run in the background. Results are stored in a folder whose name is stored in a database table; when the planner then comes to press Optimise, the system checks if there have been any data corrections – if so, it runs the optimisation from scratch as usual (the pre-emptive result for that region is thus never referenced); however, if there are no corrections, the system simply uses the stored result from the folder.

The end result for the user is that in many cases the optimisations appear to run almost instantaneously. There are really no downsides as we do not pay for our servers on a CPU cycle basis, so we can easily be wasteful of our server CPU time and run these optimisations even if their results are sometimes not needed.

One “wrinkle” we discovered with this is that we had to make our process checking more robust. There is Javascript in our browser front end that polls for certain events, such as an optimisation finishing, which is indicated by a process ceasing to exist. The Python for this is shown below, where “pid” is a process ID. The function returns True if the given process has finished or not.
def check_pid_and_return_whether_process_has_finished(pid):
if pid and pid > 0:
multiprocessing.active_children() # reap all zombie children first; this also seems to pick up non-children processes
try:
os.waitpid(pid, os.WNOHANG) # this reaps zombies that are child processes, as it gives these processes a chance to output their final return value to the OS.
except OSError as e:
if int(e.errno) <> 10: # the 10 indicates pid is not a child process; in this case we want to do nothing and let os.kill be the function to throw an exception and return True (indicating process is finished).
return True
try:
os.kill(pid, 0) # doesn’t actually kill the process, but raises an OSError if the process pid does not exist; this indicates the process is finished. Applies to all processes, not just children.
except OSError as e:
return True
else:
return True
return False

Note the “reaping” of zombie processes here, and the complication that arises if a process is not a direct child of the calling Python process (it might be a “child of a child”). In this case we use a call to the (in this case rather mis-named) function os.kill.

js/css resource serving in python apps with Fanstatic

July 27, 2011/0 Comments/in Blog, Programming, Technology /by Ash Nelson

I’ve just been checking out Fanstatic, a resource publishing/static file serving solution for wsgi python apps. I’ve been contemplating something like this as our javascript and css dependencies are getting more complex. It would also be useful to have some form of automatic cache invalidation so users don’t have to do a special browser refresh when we update our applications.

It’s easy to set up with CherryPy

from fanstatic import Fanstatic

if __name__ == "__main__":

    app = cherrypy.Application(Root())
    app.wsgiapp.pipeline.append(('repoze.who', setup_auth))
    app.wsgiapp.pipeline.append(('beaker', setup_session_storage))
    app.wsgiapp.pipeline.append(('fanstatic', Fanstatic))
    cherrypy.quickstart(app, config='workbench.conf')

after that is done, you can jquery.need() in the widget/template that needs jquery, and similar for our other dependencies. Has anyone else used fantastic? What are other solutions to dependencies and serving js/css? Is there an easier and better solution? Wrapping new libraries for fanstatic looks like a bit of effort but I haven’t explored it much yet.

Loki

Biarri provides Network Design Optimisation to NBN Co.

June 23, 2011/0 Comments/in Blog, Business /by Ash Nelson

NBN Co today announced that they awarded an 8 year contract to Biarri to supply network design optimisation to support development of efficient, lower cost network construction plans.

Biarri is to provide optimisation technology to NBN Co for fibre network design optimisation. The optimisation engine quickly generates low-cost fibre network designs based on the requirements of the reference architecture. It can determine optimal fibre area boundaries, the position of fibre hubs, and the layout and route of distribution and local fibre.

See the NBN Co site for more detail.

Logic of the Fibre Optic Network Design Tool

Tech Start Up Business Lessons

October 11, 2010/0 Comments/in Blog, Business, Other /by Ash Nelson

Biarri commenced as a commercial maths start up almost two years ago. In this time we have learnt a lot.
We focus hard on deploying optimisation and quantitative analytics in accessible ways to deliver the power of mathematics quickly and cheaply. Our just launched workbench (www.biarriworkbench.com) is a great example of this – providing monthly low cost rental of powerful maths engines available over a browser.

While we have been building products and models we have also been building a business and have learnt a few things along the way. Below are a few of the business lessons we have learnt growing a tech start up in Australia.

Be in the cloud – because we were delivering our optimisation workbench using the cloud, we sought out cloud services for our internal business needs. Our accounting software, CRM, email and timesheets are all rented from Software as a Service companies. We learnt a lot about what makes a good web app by using these services and we saved a lot of capital cost upfront. Specifically let me say that SAASU (www.saasu.com.au) is a really good accounting system for a small business – much easier to use than MYOB or quicken in my view.
Always push back against one-sided contract terms from big corporates – we find almost always you will get at least what you ask for. In house lawyers and legal departments will always try it on, especially when they are dealing with a small business – push back hard there is always some flex
Not all phone companies are the same – one large Australian telco sells conference calling enabled handsets while their network does not support conference (e.g. 3 way) calling. This is not disclosed up front- we found out when we tried our first conf call. Ask the question, be wary of penguins and remember Skype is your friend.

Hope these few thoughts help. There is more we are learning each day so will stick up some more thoughts soon.

Cross-platform development

October 4, 2010/0 Comments/in Blog, Programming, Technology /by Ash Nelson

During the course of developing Biarri’s flagship Workbench product, we’ve taken pains to ensure that our (GUI-less) optimisation “engines” work well under both Windows and Linux operating systems (so-called cross-platform). This turns out to be relatively easy as long as you stay away from the big OS-specific frameworks (e.g. Microsoft’s MFC/COM/ATL etc). We’ve picked up some handy tips along the way, particularly applicable to C++ development, which are worth sharing here.

Be aware of differences in line endings – Windows uses carriage return and line feed \r\n, while Linux/Unix uses just line feed \n. (Note that Visual Studio will show files with Linux line feeds correctly, but Notepad won’t – this is one way to tell what line endings your file has in Windows). This can be particularly important when importing data e.g. into databases where the file originates from another OS.

Always use forward slashes for file paths, not backslashes. Also, file names and folder paths are case sensitive under Linux but not under Windows. And don’t assume there is a C: or D: drive!

You may have to be careful writing to temporary files and folders. In Linux /tmp is often used; in Windows /[user]/AppData/local/temp (location of the TEMP environment variable – e.g. type “%TEMP%” into the start menu or Windows Explorer). For Linux, it is sometimes necessary to manipulate a folder’s “sticky bit” to ensure that the folder is accessible by other users (e.g. a Postgres database user) – e.g. in Python:

Be aware of the differences in file permissions in Windows and Linux. In Linux files have an “executable” bit. chmod a+x [file] makes a file an exe, which can then be run with “./filename”.

For C++ development:

Name all cpp and h files in lower case if possible. Files are case sensitive in Linux and this includes #include’s!

For compiling with GCC under Linux, the last line in a C++ file must be blank.

In Linux C++ programs, general exception handling with catch(…) does not work. You can use sighandlers instead (see this for example), though it’s not as good – it is more equivalent to an exit(), with a chance to clean up.

Beware doubles comparisons and inequality checking, at least in C++ programs. Always use a delta i.e. A == B may not be the case in both Windows and Linux if they are essentially the same number so use fabs(A – B)

Build tips for Linux: Type “make” when you are in the directory to build the project. This will search for a file called “Makefile” and run it. (Use “make -f filename” to make from a different makefile). To force a recompile you can “touch” a file using “touch filename”.
To clean out all object files type “make clean” (as long as your make file defines what cleaning does…). Use “make -j4” to run make with for concurrent jobs, to take advantage of multicore.

In bash, to get a recursive line count of .cpp/.h files: find [directory] -type f -name *.cpp -exec wc -l {} \; | awk ‘{total += $1} END{print total}’

Differences for GCC 4.3 compiler (e.g. compared to version 4.2.4): http://gcc.gnu.org/gcc-4.3/porting_to.html. (To check your version of GCC, type gcc -v).

The Launch of Biarri’s WorkBench

September 29, 2010/0 Comments/in Blog, Business, Operations Research, Technology /by Ash Nelson

With the impending launch of Biarri’s workbench and our ongoing close relationship with Schweppes for the daily routing of soft drink deliveries (an application of perhaps the most well known operations research problem: the vehicle routing problem), I thought that the following excerpt from a journal article submitted to the Asia Pacific Journal of Operations Research would be a very timely blog post.

The journal article is entitled “Real-Life Vehicle Routing with Time Windows for Visual Attractiveness and Operational Robustness” and it describes the vehicle routing algorithm we have implemented for Schweppes.

The excerpt details a specific example encompassing two things we are very passionate about at Biarri. First “Commercial Mathematics” – that is making OR (well not strictly just OR) work in the real world. And second, the revolutionary capabilities that the advent of cloud computing has for the delivery of software.

“Vehicle routing problems manifest in a remarkably wide range of commercial and non-commercial enterprises. From: industrial waste collection to grocery delivery; underground mining crew replenishment to postal and courier collection and delivery; inbound manufacturing component transportation to finished car distribution; in-home primary health care delivery to pathology specimen clearances from surgeries for analysis; and from coal seam gas field equipment maintenance to beverage distribution, to name but a few.

Automated planning systems used by industry at present are predominantly client-server or desktop based applications. Such systems are often: expensive, requiring a large upfront capital investment; accompanied by a large software deployment project requiring initial and ongoing IT department cooperation; customisable to a particular organisations requirements, however commonly retain a large amount of exposed functionality due to the breadth of the existing client base; and require substantial user training as the workflow is usually not restricted in a linear fashion …. Each of these characteristics constitutes a barrier to adoption of automated planning systems, and for most small to medium enterprises these barriers prove insurmountable.

With the advent of cloud computing and software as a service (SaaS) these barriers are being removed. SaaS: embodies a different commercial model; has essentially no IT footprint; mandates (as vendors may never directly interact with potential clients) simple intuitive linear workflows; and involves almost no end user training beyond perhaps an optional demonstration video.

The emergence of this new avenue for the delivery of optimisation based planning systems heralds, a heretofore, unparalleled opportunity for operations research practitioners to engage with a wider potential consumer base than ever before. However, the nature of the delivery mechanism requires the algorithms developed: to be robust and flexible (within their domain of application they must be capable to dealing with a wide range of input data); to have very short run times (the user base is more likely to be under time pressure than ever before); to produce high quality solutions (noting the inherent trade off between run time and solution quality); to be wrapped in a simple linear workflow (meaning it is always obvious what the next step in the planning process is); but above all, be able to produce real-life, practically implementable solutions, without the need for user training and/or experience.

For pure delivery, or pure pick up vehicle routing applications, real-life, practically implementable solutions are often synonymous with geographically compact, non-overlapping routes with little or no intra-route cross over. There are numerous reasons why such solutions are preferred …. If a customer cannot be serviced at the preferred time (e.g. the vehicle cannot get access, the customer is closed, another delivery is taking place, the customer is too busy), because the route stays in the same geographical area, it is easy to return to the customer at a later time. During busy traffic periods drivers are loathe to exit and re-enter a motorway to service individual customers. Even though such customers may be enroute to the
bulk of the customers the route services, thus incurring a minimum of additional kilometres, they may nevertheless be far from the majority of the customers the route services. If there is severe traffic disruption, it is easier to use local alternate routes between customers in a route that is geographically compact to ensure that pick-ups or deliveries can still be made. Third party transport providers, which prefer routes to be as simple as possible, may exert some influence over the planning process. Finally … it is easier to maintain customer relationships by assigning drivers to routes that routinely service a similar geographical area. In summary, solutions which are more visually attractive are more robust, and thus more likely to actually deliver the full extent of the cost savings that should flow from the use of automated planning systems.

This paper describes an algorithm for the vehicle routing problem with time windows, …. The algorithm is: robust and flexible; fast; wrapped in a user interface utilising a simple linear workflow and so requires no user training or experience; and produces high quality, visually attractive and practically implementable solutions.”

Posts