We ended up pulling together 12 different stocks from companies that in one way or another practice architecture, engineering, and/or construction. If you know how much of this market is privately owned you are probably as surprised as we were that there were even 12 of these companies to find.

The results are pretty interesting and it’s too bad this isn’t a real stock. If you invested $1,000 back in January of 2006 you would have more than doubled your money by August of 2014 which seriously outperforms the general market. 2010, ’11, and ’12 all show what appear to be seasonal dips while 2013 was nothing but gains. The jury is still out on 2014 but our money says that the market is ripe for another small dip.

In the coming weeks we’ll be looking to automate the construction of this index which will enable us to deliver a daily market indicator to our readers. Stay tuned.

In this follow-up we’ll give some background on our “Data Practicality Framework”. We’ll show how it can be used by applying it to the American Institute of Architects’ Architectural Billing Index.

#1: Where does it come from?

This is another way of saying “consider the source”. If someone has a stake in the information they’re presenting there is a good chance that they are putting a particular spin on it. We can’t even blame anyone for this. It’s a result of the human factor. Information that is entirely free of bias is not only difficult to present, it’s boring. Human beings end up unconsciously promoting things they want to be true and hiding things they don’t want to admit. The numbers don’t lie but the people presenting them might.

#2: How is it measured?

Answering this question can help us figure out if we’re looking at a parameter or a statistic. A parameter is the most pure form of data because it represents an entire population. We are not very likely to see true parameters in our everyday experience. A statistic attempts to represent a much larger data set by taking a random sample and assuming it has the same characteristics as the total population. Generally speaking bigger samples are better. Bigger samples are harder to get which is part of where big data comes in. Sometimes practicality gets in the way of collecting big samples. We wouldn’t be talking about big data if it wasn’t getting easier but some metrics will always be estimates.

#3: Who is it meant for?

We’re assuming that you aren’t using the price of corn futures to project how your AEC business is going to do next month. If you are then maybe you should answer this first. Either way, we suggest that you ask this question at some point because the data might not be as relevant as you think. We aren’t completely discounting indirect data sources but given the lack of engineer and consultant specific data it bears consideration. If your firm specializes in lighting design for greenhouses you very well might benefit from agricultural data. But if you do use indirect data it’s important to understand what makes it indirect. Will your business lag the data by a few months? Maybe you are operating in a subset of the overall market? Considering these differences will help to understand what’s really impacting your business.

#4: How can I use it?

This is where we can start to leverage the whole process. The answer to this question depends on the type of data you have and what you’ve gotten out of it. Putting some serious thought into what you can do with the data could mean the difference between having it positively affect your business or not. It’s tempting to ask this question first but saving it for last prevents us from using biased, poorly measured, unrelated data for something we shouldn’t. Let your mind wander with this one but make sure you come out of it with some concrete action items. Maybe pick a time on the calendar when you can look back and see if the suggested outcome actually happened?

Now it’s time to apply this to the ABI and see what it is/isn’t good for.

Data Practicality Framework And The ABI

#1: Where does it come from?

The ABI is generated by the AIA Economics and Market Research group. This group is part of a professional organization that is polling Architect’s on a monthly basis to see what is happening with their business. In this context there’s really nothing objectionable about it. The ABI is probably as free of bias as any research can be. This is still a good question to ask because a “for profit” organization might be motivated to paint a more promising picture of the economy. We could go into scores of examples where editorial commentary has downright abused the use of statistics and data so the point is, always proceed with caution.

#2: How is it measured?

The ABI is a statistic and not a parameter. We know this because the AIA tells us there are approximately 700 architecture firms surveyed for this study. Earlier studies included nearly 300 respondents. So not only are we looking at a sample but we’re looking at one that has changed over time. If we compare this to the AIA estimate that 17,500 firms are owned by AIA members we see that the ABI is sampling at most, 4% of the industry (there are probably more than 17,500 firms). Statistically 700 is a great number. If the firms are randomly chosen there’s a strong chance that it actually is indicative of architect’s as a whole. But there’s one characteristic of the ABI that we know makes it non-random. The survey is only a product of those firms who respond. Is there some other characteristic of these responding firms that biases the data? Maybe smaller firms don’t contribute suggesting that this is more applicable to larger firms?

Another aspect of the ABI is unfortunately more disappointing. The Economics and Market Research Group isn’t secretive about this. They come out and say it right on their website. However, most people may not consider the result of this survey method. The ABI is a “diffusion index”. Survey respondents are given three questions: have billings grown more than 5%, have billings remained the same, have billings declined more than 5%. If 50% say growth and 50% say decline the score will be a 50. If 100% say stable the score will also be a 50. If 100% say growth the score will be 100. The problem we have is that the score can say 100 but the market will not have grown 100%. It has the effect of making the numbers look like a stock market index for architects, which it’s not. This is why we think it’s misleading. It’s not worth feeling cheated over because it’s not intentional but if you don’t know about it you might draw some erroneous conclusions.

#3: Who is it meant for?

This should be obvious. The ABI is meant for architects. It’s not meant for engineers or consultants. The ABI has been featured in a study published by Business Economics showing it’s relevance to construction cycles which is encouraging. It is also probably a leading indicator because architect’s have to get paid before Es and Cs do. But how much does it actually lead your business? Hard to say.

#4: How can I use it?

This is a tough one. Based on everything we’ve covered, Engineers and Consultants should see the ABI as a pulse on the status of architects and nothing more. Some businesses may not see their billings change at all in relation to the ABI. Others may see a direct correlation. What we can say with certainty is that it won’t have the same significance for everyone.

Hopefully you feel a bit more comfortable looking at new sources of data by using this framework.

Whether you’re digging it out of your own marketing database, reading it in a magazine, paying a hefty fee, or paying even more for a custom study, getting data is never simple. Perhaps one of the most frustrating parts of being a manager at an Engineering or Consulting firm right now is that everyone in the business world is talking about “Big Data” and it doesn’t seem like it relates to you. What is a manager to do? Here are some ideas.

Internal Data

The best way to start is by looking in the mirror. What do you want to know about your own company? Most mid to large size companies have databases for marketing or accounting. Just because you’re a one man shop doesn’t mean you can’t do this too. Everyone should be tracking their own data no matter how big or how small. If you aren’t doing it now we can’t recommend enough that you start. Excel and Quickbooks are perfectly fine places to begin. Throughout our experience we’ve identified over 25 different fields that anyone handling proposals can track. Here’s a handful to get you started:

Once you have these fields you can start asking questions like “How much of our business is done directly with Owners, Architects, or Developers?” You might have an intuition for that but it’s always better to use hard numbers. Particularly because you can see how they change over time.

You don’t have to rack your brain coming up with as many “fields” as you can think of either. Different organizations will have different data needs but it’s important to be practical about it. It’s also important to start somewhere. The benefits of tracking data can’t be overstated enough and the sooner you start the better. Knowing that you landed 5 projects last month will push you to go for 6 this month and 7 the month after that.

Lastly, if you have the feeling that you need to start with all the right fields you should ignore it. When you find out two years later that you wanted to track different market sectors it won’t mean the old data is useless. You can even go back and update the old data to get what you want from it. This is a great way to give old data new life and get more from it. Sure, it’s a pain. But it’s better to have data you can work with then nothing.

External Data

If you already track loads of data internally this part is for you. It’s where things are getting interesting. Internal data is great but it only allows us to analyze things we think we already know. Data from the outside world puts that internal data into perspective.

Even small bits of external data can be useful. For example, if we look at the AIA website we see that there are 17,500 architecture firms owned by AIA members. For the larger firms out there; do you know the number of different clients that you have worked for? What percentage of this 17,500 have you had access to? How has your number of clients changed? Okay, “Small Data” isn’t what anyone is talking about these days so what are some more sizeable options?

The ENR Top 500

We have blogged this one to death already but it bears repeating. The ENR 500 is the easiest “go-to” list of the biggest firms in the industry. For some companies this won’t be enough. For other companies it will seem outside of your reach. It’s not. This list is useful to anyone. Even if you’re a small firm, establishing a solid relationship with even one of these firms could mean the difference between celebrating at the end of the year and shutting your doors. Read the list, find the companies you know and work for, then cross them out and learn about the others.

The AIA ABI

The American Institute of Architects’ Architectural Billing Index (“ABI”) is one of the first data metrics we found when we began looking back in our early years. It’s no small wonder. The ABI has been around in one form or another since 1995 and offers what some would consider a “stock market like index” of the entire US architecture industry on a monthly basis. We will go into the ABI in more detail in Part II. Rest assured that whatever qualms we may have, the ABI is a metric you should follow and be aware of.

“Independent” Options

This is an area where it should come as no surprise that we are a little biased. There are currently a handful of different firms that cobble together studies at various times with a specific purpose in mind. We’re not going to pick on any in particular which is why we developed a list of questions. These questions form a framework that you can use to analyze any potential data source to see how it relates to your own business practice.

Data Practicality Framework

1. Where does it come from?
2. How is it measured?
3. Who is it meant for?
4. How can I use it?

Most of the independent data sources are collecting information from big companies and presenting it back to them. In that context these studies have a purpose. Smaller unrelated firms will probably find it hard to justify the cost of these reports when the titles alone are meant for someone else. Still other providers are combining so many different types of companies you have no idea if it’s actually useful.

This is where Data Cornerstone comes in. We’re able to combine data from various specialty firms and give you a metric like the ABI for your specialty discipline. Not only will we tell you what has happened, we’ll tell you where it’s going. The Data Cornerstone model relies on the knowledge that small firms make up a significant percentage of any professional services market which is exactly what these reports typically leave out. Even among architects the AIA estimates that firms with less than 9 staff make up 81% of architecture firms and 21% of architect’s billings.

Next time we’ll go into more detail by applying our “Data Practicality Framework” to the ABI. In the meantime, leave us feedback in the comments. Are there other sources of data we forgot? Maybe we can help you develop your own sources of internal data.