Nate's Stuff : Architecture

Visual NDepend Blows My Socks Off!

Nathan Zaugg — Fri, 27 Feb 2009 00:10:23 GMT

I’m not easily impressed by even the best of software. There is usually some little thing that ruins my user experience. I tell you this so you know that what I say next is something special. Visual NDepend is AWESOME! Everything meet or far exceeded my expectations! If you are not yet running this software, you should be! This is not one of those ‘nice to have’ pieces of software like you might thing. This is a must have! If Microsoft was smart they would do everything they could to package this in with Visual Studio 2010.

What does it do? It’s hard to answer that question in a paragraph or two. The basic answer is that it gives you a set of different representations of your software in such a way that you can understand the architecture and complexity. This can be a huge help before undertaking large refactor efforts. It also gives you an idea or the quality of the code based off of some metrics. This quality score can be a really good indicator of wither your code is going to be maintainable and if it’s not it can help you get it there. Before I go through the quick walkthrough of the software it’s self I’ll just say this – If you’re not paying attention to code metrics and the maintainability of the code from the earliest stages of development your likely not going to develop code that is easy to maintain. And as you probably know, software maintenance is usually more than 3/4ths the cost of the software lifecycle. Therefore, if you can get a build process established early and use a tool like NDepend to keep a close watch on some important metrics I can guarantee you will be creating much higher quality software than you would have otherwise.

Installation Experience

This is the best kind of install – you simply unzip the files. I bet you didn’t think I would do a whole section on install when there is nothing to the install. I suppose if it were any other product there would be a lot under this section.

Analyze a Project

The user interface is very intuitive. I installed integration with Visual Studio 2008 and .NET Reflector with minimal effort and started to analyze my assembly.

It took a while to analyze but this particular assembly is quite large! It also pulls in any referenced assemblies so the graph can be complete. It allows you to specify multiple assemblies just in case you have sets of assemblies in the same project that have no direct relationship. In this case I had a WCF service that would need to be added separately as it’s technically not a direct dependency. Once the analysis is complete it’s INFORMATION OVERLOAD!

Results

There are six or seven main results to view. The most obvious is the bubble chart at the top. This gives you a visual representation of different aspects of the code. For example it’s currently set to Method level and the metric shown is number of lines of code. the larger the bubble, the larger the method is in relation to the others. I can select from Method, Field, Type, Namespace or Assembly. In addition there are 11 different metrics that you can plot in the bubble chart.

The next thing you notice is the dependency graph. Unfortunately this project doesn’t have a dependency graph like in my other blog post. Visual Studio makes it really tough to make circular dependencies but it is possible to create them if not all of your projects are loaded into the same solution. And as I said in the other post, circular dependencies are very bad! It’s also good to know what references what. You might be surprised to see what kind of references get worked in over time. On a recent project the data layer code kept working it’s way up to the UI even though it didn’t belong there.

The next section is the code metric-centric part. This is my most favorite portion of the results generated and can be a real good indicator of how maintainable a codebase is. Of course it’s impossible to avoid all complexities in software so you probably can’t have a perfect score with every project but you can use this information to figure out what kinds of habits you have that contribute to un-maintainable code. As you can see from the image below when you select a metric it is highlighted in the bubble chart. In fact, all of these reports are “active” so anything that is relevant to another will be indicated. By the way, this application has amazing performance. I’m pretty sure it’s using WPF under the hood.

Another really cool feature is the Dependency Matrix. This takes some practice to use but it helps identify which pieces of which assemblies access pieces of other assemblies. Again, great for refactoring efforts and a good indication of how an application is put together. You may have to stair at the image below for a while to figure out what it’s trying to tell you but after a while it makes perfect sense.

Last, but certainly not least is the comprehensive report compiled into HTML. This is the kind of thing that I’d love to see emailed to me each day, so I hope it’s easy to put this into a build process. The report is awesome! My favorite part of the software and I really like this software. Below is the screen shot of the table of contents that is generated at the top.

The only feature I didn’t get a chance to try is code coverage results. I would have loved to see that in action! Too bad I don’t have a project with very good code coverage results and I’d be embarrassed to find out anything in detail about my code coverage. Of course code coverage isn’t everything but it is a useful tool when taken into context and can be made more useful with nDepend.

Overall, I would give this software very high marks! I can’t wait to make this part of my build process! It is worth the time and money!

The Service Oriented Architecture (SOA) Philosophy

Nathan Zaugg — Wed, 05 Nov 2008 22:53:00 GMT

Service Oriented Architecture is a software design concept that packages small pieces of modular functionality into "services" through the software lifecycle. SOA separates functionality into a set of self-contained, modular units that can be combine by a simple application to accomplish a task. This blog post will focus on a few specifics of SOA but will mostly discuss the benefits of using SOA in the Enterprise.

Background

One of the main pieces of philosophy that SOA embraces is the KISS (Keep It Simple & Stupid) principle. The fact is that big complex projects fail while small, simple projects succeed. As a developer I see this every day. For example, a few months ago me and my friend Phil wrote a web site. In the early days and weeks (after getting some of the basic requirements ironed out) functionality was really easy to add. Everything existed in a very simple state because it was a simple design with simple requirements. We finish the original set of requirements pretty early in the project when they decided that we wanted to do some enhancements. These enhancements greatly complicated the application. Now, adding even a small amount of functionality takes substantially longer. Over the weekend I wanted to add the ability to add category in as part of the search. In the early days of the project it was actually work to leave it out (I had to disable the box when they changed the type of search). It was pretty easy to add it into the stored proc but then the coding changes! I had to add it to my data layer, data model, view, and controller. Of course, these changes broke some other functionality that was added after this feature was originally included. Because Unit Testing wasn't going to be part of the timeline we have to test a large portion of the site to be comfortable that this change didn't break anything else. The two universal facts are that:

The larger and more complex an application becomes the longer it takes to add features.
Changing code is more expensive that writing new code.

In Juval Lowey book called Programming .NET Components he highlights that component-oriented programming can simplify and add flexibility to complex applications. He also makes a good case for simple software and all of these concepts port directly into the SOA paradigm.

What is a service?

When many people hear the term "service" they almost always either think of Web Service or Windows Service. The truth is that a service can be almost anything that provides implementation to a consumer. In this context a service can be a library that can be invoked (much like Win32 API), or it could be a piece of hardware that performs complex calculations (like a Graphics Card/GPU). Believe it or not the World-Wide Web is just a bunch of different services. Web services have been particularly popular as they meet the criteria for usability. 1) Universally accessible, 2) Simple and Stateless. In addition web services are easy to deploy, change, and have an already extensive existing infrastructure. Web Services are also very scalable and are easy to make redundant which provides high availability.

Figure 1: Enterprise SOA Diagram

Why SOA?

Besides the benefits from rapid software development there are many other benefits of SOA. For those looking to make a business case of ROI consider that SOA helps to make your company easily and quickly change business processes to align with market conditions. This is because the software is now modular and flexible and especially if you use some type of workflow for your complex business tasks these rules can be easily updated and changed. Also, new functionality is quickly developed, integrated, and deployed. Consider the difficulty involved in making a change in the order in which orders are filled. With a Fat-Client approach you may have several different apps that depend on the order being predictable.

Testability

Testability is arguably one of the best benefits gained by the SOA approach. Because we have a simple granular piece of functionality it makes testing of this functionality easy. It is also easy to test as these smaller functions are designed to be exposed. It can be difficult to test code that wasn't intended to be consumed. In terms of change management, if we keep our business running in services then our unit tests can help guarantee confidence in a new release of an application that uses several different services.

Because the database is only touched by one layer of our enterprise application infrastructure and due to the fact that most services will exclusively deal with a small subset of tables -- changes to the database become greatly simplified in a true SOA environment. This adds simplicity and allows the company to change more quickly to adapting demands on their business. In addition different systems can be allowed to run on different databases which makes migration from one database to another significantly simpler.

Security

Security is a great reason to move toward SOA. The biggest gain is that the Database servers can be safely tucked in a Back End Zone (BEZ) because our applications no longer need to connect directly to them. This is also extremely helpful if you have each user connect to the database using Integrated Authentication or an account unique to each. This will cause hundreds and thousands of connections on the database and reduce performance substantially! Moving the database into a zone where it can only be accessed from the application servers limits the surface area for attack and greatly limits potential venerability. For financial systems it is part of the infamous Sarbanes-Oxley (SOX) audit to prohibit access directly to the database (in addition to change logging requirements). While this is not the only way to come into compliance with a SOX audit it is arguably one of the better ways.

Simplicity

Again, the best benefit of SOA is the simplicity each service can have which results in less bugs, more developer productivity, modular reusable code, and flexibility. Services should be broken up into a lot of different chunks to make them easier to manage and they should typically include only one concern (a single aspect of the business). Other services can consume existing services to expose more complex business sceneries.

"It breaks down the complexity of software systems by isolating services. This simplifies configuration management for each piece, and whole applications can be made compliant with new ordinances and business requirements without updating any of them." -- Phil Gilmore

Another benefit SOA gives us is that every call needs to have an interface that is carefully and purposefully designed. Using Interfaces is a great way to promote loose-cohesion and makes changing different parts of the code much easier to accomplish. So long as any changes you make can still be supported by the interface already defined most changes are inconsequential. If you must make a change to the interface of a service you will typically be doing this to add new features and usually the existing interface can be left alone and a new version of the interface can be supported for that service.

"SOA has a downside in version control. Services must usually be able to run side-by-side versions to support applications whose requirements don't change synchronously." -- Phil Gilmore

Performance

Performance is an important consideration when talking about services (especially web services) and potential impact to application performance. Network round-trips are expensive -- even in a LAN environment. Also the serialization and de-serialization of objects adds extra time for each call -- time that is not necessary when using a framework-style call. While all of this is true, consider that any data retrieval is expensive. If your application connects to a central database you still incur the overhead of connection to the database over the network, your data is still serialized as it goes over the wire and you will usually still hydrate objects with this data once you get it back. At that point the only layer that a web service has that is extra is that of IIS. When you factor in the connection pooling that can be accomplished on a service with the cost of having many (and perhaps even thousands) of concurrent connections to the database when thick clients connect directly it's easy to see that your probably not going to see any significant performance loss by moving to this type of architecture. It may take some getting used to when creating calls with a ton of parameters but it will perform well.

Another thing to consider when looking at performance aspects of SOA is that a web service written with WCF is fast! How fast? I haven't benchmarked web services in .NET for quite a while (since Framework 1.1), but I know there were significant performance gains for the 2.0 Framework and even more gains for 3.0 & 3.5 Frameworks. Additionally, the newer frameworks have much lighter formatters and endpoints than we did back then. Keeping this in mind, back when I benchmarked a simple web service that would reverse a string that was sent to it as input that service was ran at a rate of 2,200 calls per second sustained! This is a testament both to IIS as a middleware server and .NET Web services for sheer speed! Our other web service was coded in Delphi 7 as an ISAPI DLL and struggled to sustain 10 calls per second. The Delphi service also intermittently failed whereas the .NET web service did not. In short, the web is a great way to expose a service and there is little need to look for anything "faster" or "more robust".

Configuration Management / Distribution

When looking to other positive aspects of SOA, without a doubt one of the best arguments is distribution. The race to "web enable" everything possible is a landmark of the difficulties of distribution. These so-called thin clients relegate any deployment and configuration to the server that runs the service. This concept of "configure once" is embraced by SOA. If you plan on deploying a smart client (a cross between a thick client and a thin client) significantly reduces the amount of configuration required for users to execute your software. They don't have to worry about database providers, connection strings, loggers, DSN's, or anything else your app might use. SOA can even be used for web sites (thin clients) that require these services. SOA is a principal without a fixed style of client so it will work as well with a web app as it will with a windows app.

"This saves configuration management by avoiding redeployment of existing applications, it saves software engineering by not modifying the applications, it tends to save time in testing if testing can be isolated to that service rather than testing all applications that consume it." -- Phil Gilmore

Summary

Service Oriented Architecture is a best-practice for today's Enterprise. It's use can substantially reduce the manpower required to maintain applications and has major improvements to maintenance, scalability, change management, code reuse, and distribution of applications. SOA will make your life easier and make your employer happy and isn't that the most important argument of all?

Google releases web browser "chrome"

Nathan Zaugg — Tue, 02 Sep 2008 23:30:00 GMT

Google has done it again! In a completely unexpected (at least by myself) move they have released a browser into mainstream. Frankly, I am quite surprised! Google was such a fan of Firefox that I never thought I would see them in competition. It also surprises me because building web browsers has is not part of Google's core business. This is also an obvious swipe at Microsoft which doesn't excite me. A friend of mine upon hearing the news said "I don't like the idea of any major web presence building a major web browser and driving a supposedly "open source" technology to do what suits them best."

Install

I have no idea how large the install really is. The download is a tool that downloads the rest of the browser. It downloaded for a few min's and began the install. My estimation was it was probably about 10-30MB though I really don't know. The installation was very uneventful.

What Google did right

There are a few things I really like. Here they are:

The interface is nice, sleek, and small. I hate browsers (Firefox, you know who you are) that take up TONS of vertical space on my screen! The space is wasted with things I don't even need or want! Google's chrome did a nice job here!
I like the idea of a multi-threaded browser (even if it is a multi-process browser).
JavaScript is now a 1st class language in chrome. V8, the JavaScript execution engine It is runs JavaScript much the same way as managed code being JIT'ed directly to the CPU. As a result, it is LIGHTNING FAST!
I really like the 'URL Bar'. It has a suggest/append that actually gets what I want! I also like the landing page.
I like what they did to get the "sandbox" mode. I especially like the way it's transparent to the user what is crashing your browser (usually a plug-in).
I like the incognito feature which allows zero trace browsing. I don't know why we don't do this all of the time!
The browsing experience is very fast. The browser UI is also fast.
I like the tab reordering. It's very smooth!
I think the "move tab to another Google browser window" is cool but ultimately not very useful.
I like the "task manager"

What Google didn't do right

The multi-process approach is a turn off. This means that they have to have a JavaScript rendering engine in memory for each process, it means that things like settings, etc exist in each process. Most of all processes are EXPENSIVE! Very much overkill! Yes, I know that this is how IE does it, and I've never liked it!
The browser is too simple and has very little customization out of the box.
It uses a *lot* of system resources! Especially Memory!
I do not like that it uses a lot of the same code as Safari! SAFARI NEEDS TO DIE! IT SUCKS!!! HOW MANY YEARS DO WE HAVE TO WAIT FOR HTTP1.1 PROGRESSIVE DOWNLOADING? This is a big deal in my opinion.
It has a book marking system that claims to be different but sure looks exactly like IE7 bookmarks to me!
It was using a lot of my CPU! Something like 60% of my CPU was going toward chrome! I may have had a malfunctioning plug-in but even when I closed the tabs that looked like the culprit it was still using 11-17% of my CPU!
It didn't seem to start up any faster than IE8 or FF3.
Google didn't release anything until now. So far as I know there wasn't even a beta version for users to try. I looked for some plug-ins but couldn't find any. A little heads up to the development community could have had many plug-ins available by the time the browser launched.
I know this is a small thing, but I hate when the browser "styles" text boxes for me! What if I wanted to implement my own highlighting? What if I didn't want it to highlight?
It would lie to me about processor usage! I had a process that was using 45% of my CPU and that none of the processes claimed to take that much CPU. This was a Gmail tab, ironically!
Silverlight didn't work. It tired, but never really loaded.

I've never been comfortable with Google's privacy policy. The fact that it reads my G-Mail and suggests products much like the topics inside rubs me the wrong way. The fact that they would like to track everything you do including every web page you visit SCARES ME!

Anyway, here are some JavaScript comparisons. I only have IE8 (which isn't going to be ready for such tests) so those tests aren't very relevant.

Google vs IE 8 (in IE 7 Emulation Mode)

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           4.93x as fast     18324.4ms +/- 1.9%   3716.2ms +/- 3.5%     significant

=============================================================================

  3d:                  6.99x as fast      1816.8ms +/- 4.1%    260.0ms +/- 11.7%     significant
    cube:              10.5x as fast       645.2ms +/- 5.8%     61.6ms +/- 8.8%     significant
    morph:             3.96x as fast       481.8ms +/- 11.0%    121.6ms +/- 21.7%     significant
    raytrace:          8.98x as fast       689.8ms +/- 3.4%     76.8ms +/- 18.2%     significant

  access:              19.5x as fast      3398.4ms +/- 1.3%    174.6ms +/- 11.1%     significant
    binary-trees:      47.3x as fast       482.4ms +/- 5.7%     10.2ms +/- 5.5%     significant
    fannkuch:          18.2x as fast      1199.0ms +/- 1.4%     65.8ms +/- 29.0%     significant
    nbody:             21.6x as fast      1322.4ms +/- 1.5%     61.2ms +/- 14.0%     significant
    nsieve:            10.6x as fast       394.6ms +/- 8.9%     37.4ms +/- 17.0%     significant

  bitops:              18.4x as fast      2197.0ms +/- 3.1%    119.2ms +/- 11.7%     significant
    3bit-bits-in-byte: 38.1x as fast       327.6ms +/- 7.1%      8.6ms +/- 7.9%     significant
    bits-in-byte:      24.5x as fast       485.2ms +/- 3.2%     19.8ms +/- 19.6%     significant
    bitwise-and:       25.4x as fast       858.2ms +/- 2.8%     33.8ms +/- 9.5%     significant
    nsieve-bits:       9.23x as fast       526.0ms +/- 7.3%     57.0ms +/- 19.9%     significant

  controlflow:         77.7x as fast       435.0ms +/- 1.8%      5.6ms +/- 12.2%     significant
    recursive:         77.7x as fast       435.0ms +/- 1.8%      5.6ms +/- 12.2%     significant

  crypto:              10.8x as fast      1176.4ms +/- 3.6%    108.6ms +/- 12.5%     significant
    aes:               12.2x as fast       518.8ms +/- 9.1%     42.6ms +/- 11.4%     significant
    md5:               8.04x as fast       318.4ms +/- 6.2%     39.6ms +/- 33.3%     significant
    sha1:              12.8x as fast       339.2ms +/- 7.3%     26.4ms +/- 9.8%     significant

  date:                1.87x as fast      1739.4ms +/- 5.1%    928.4ms +/- 6.6%     significant
    format-tofte:      1.57x as fast       733.8ms +/- 6.9%    468.6ms +/- 6.0%     significant
    format-xparb:      2.19x as fast      1005.6ms +/- 8.6%    459.8ms +/- 9.9%     significant

  math:                6.73x as fast      1607.6ms +/- 4.9%    239.0ms +/- 15.2%     significant
    cordic:            4.79x as fast       676.4ms +/- 5.9%    141.2ms +/- 27.4%     significant
    partial-sums:      5.98x as fast       443.4ms +/- 4.3%     74.2ms +/- 18.6%     significant
    spectral-norm:     20.7x as fast       487.8ms +/- 12.2%     23.6ms +/- 19.9%     significant

  regexp:              *1.07x as slow*     719.6ms +/- 3.6%    767.2ms +/- 3.3%     significant
    dna:               *1.07x as slow*     719.6ms +/- 3.6%    767.2ms +/- 3.3%     significant

  string:              4.70x as fast      5234.2ms +/- 2.4%   1113.6ms +/- 5.4%     significant
    base64:            15.8x as fast      2212.6ms +/- 4.4%    139.6ms +/- 20.7%     significant
    fasta:             9.57x as fast      1117.2ms +/- 2.8%    116.8ms +/- 7.5%     significant
    tagcloud:          1.86x as fast       597.0ms +/- 3.4%    321.4ms +/- 8.5%     significant
    unpack-code:       1.82x as fast       710.8ms +/- 3.8%    391.2ms +/- 3.3%     significant
    validate-input:    4.13x as fast       596.6ms +/- 1.8%    144.6ms +/- 10.4%     significant

Google vs Firefox 3

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           1.65x as fast     6138.2ms +/- 27.7%   3716.2ms +/- 3.5%     significant

=============================================================================

  3d:                  3.31x as fast      860.8ms +/- 66.9%    260.0ms +/- 11.7%     significant
    cube:              -                  422.8ms +/- 114.5%     61.6ms +/- 8.8% 
    morph:             1.68x as fast      204.6ms +/- 1.6%    121.6ms +/- 21.7%     significant
    raytrace:          3.04x as fast      233.4ms +/- 40.0%     76.8ms +/- 18.2%     significant

  access:              6.01x as fast     1049.8ms +/- 31.9%    174.6ms +/- 11.1%     significant
    binary-trees:      -                  185.2ms +/- 168.8%     10.2ms +/- 5.5% 
    fannkuch:          7.25x as fast      477.2ms +/- 8.9%     65.8ms +/- 29.0%     significant
    nbody:             3.98x as fast      243.4ms +/- 11.2%     61.2ms +/- 14.0%     significant
    nsieve:            3.85x as fast      144.0ms +/- 10.4%     37.4ms +/- 17.0%     significant

  bitops:              5.52x as fast      657.8ms +/- 2.2%    119.2ms +/- 11.7%     significant
    3bit-bits-in-byte: 13.2x as fast      113.4ms +/- 7.2%      8.6ms +/- 7.9%     significant
    bits-in-byte:      8.76x as fast      173.4ms +/- 6.4%     19.8ms +/- 19.6%     significant
    bitwise-and:       4.44x as fast      150.0ms +/- 12.0%     33.8ms +/- 9.5%     significant
    nsieve-bits:       3.88x as fast      221.0ms +/- 2.5%     57.0ms +/- 19.9%     significant

  controlflow:         12.8x as fast       71.8ms +/- 5.1%      5.6ms +/- 12.2%     significant
    recursive:         12.8x as fast       71.8ms +/- 5.1%      5.6ms +/- 12.2%     significant

  crypto:              3.29x as fast      357.0ms +/- 6.3%    108.6ms +/- 12.5%     significant
    aes:               3.37x as fast      143.6ms +/- 11.4%     42.6ms +/- 11.4%     significant
    md5:               2.73x as fast      108.2ms +/- 9.9%     39.6ms +/- 33.3%     significant
    sha1:              3.98x as fast      105.2ms +/- 3.1%     26.4ms +/- 9.8%     significant

  date:                *1.73x as slow*    537.8ms +/- 43.8%    928.4ms +/- 6.6%     significant
    format-tofte:      ??                 359.8ms +/- 56.1%    468.6ms +/- 6.0%     not conclusive: might be *1.30x as slow*
    format-xparb:      *2.58x as slow*    178.0ms +/- 19.3%    459.8ms +/- 9.9%     significant

  math:                2.59x as fast      619.0ms +/- 12.5%    239.0ms +/- 15.2%     significant
    cordic:            1.93x as fast      272.6ms +/- 6.1%    141.2ms +/- 27.4%     significant
    partial-sums:      3.14x as fast      233.0ms +/- 35.7%     74.2ms +/- 18.6%     significant
    spectral-norm:     4.81x as fast      113.4ms +/- 9.1%     23.6ms +/- 19.9%     significant

  regexp:              *1.68x as slow*    457.0ms +/- 7.7%    767.2ms +/- 3.3%     significant
    dna:               *1.68x as slow*    457.0ms +/- 7.7%    767.2ms +/- 3.3%     significant

  string:              -                 1527.2ms +/- 37.5%   1113.6ms +/- 5.4% 
    base64:            -                  145.0ms +/- 12.2%    139.6ms +/- 20.7% 
    fasta:             3.01x as fast      351.2ms +/- 49.9%    116.8ms +/- 7.5%     significant
    tagcloud:          ??                 311.8ms +/- 71.6%    321.4ms +/- 8.5%     not conclusive: might be *1.03x as slow*
    unpack-code:       1.28x as fast      501.0ms +/- 14.4%    391.2ms +/- 3.3%     significant
    validate-input:    -                  218.2ms +/- 54.4%    144.6ms +/- 10.4%

Google vs IE 8

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           4.70x as fast     17474.2ms +/- 1.6%   3716.2ms +/- 3.5%     significant

=============================================================================

  3d:                  6.87x as fast      1786.8ms +/- 1.8%    260.0ms +/- 11.7%     significant
    cube:              10.1x as fast       620.0ms +/- 5.4%     61.6ms +/- 8.8%     significant
    morph:             3.86x as fast       469.6ms +/- 6.5%    121.6ms +/- 21.7%     significant
    raytrace:          9.08x as fast       697.2ms +/- 4.1%     76.8ms +/- 18.2%     significant

  access:              16.3x as fast      2847.8ms +/- 1.7%    174.6ms +/- 11.1%     significant
    binary-trees:      49.4x as fast       504.2ms +/- 8.6%     10.2ms +/- 5.5%     significant
    fannkuch:          18.5x as fast      1219.8ms +/- 6.5%     65.8ms +/- 29.0%     significant
    nbody:             11.6x as fast       712.0ms +/- 6.8%     61.2ms +/- 14.0%     significant
    nsieve:            11.0x as fast       411.8ms +/- 14.2%     37.4ms +/- 17.0%     significant

  bitops:              19.0x as fast      2268.8ms +/- 6.8%    119.2ms +/- 11.7%     significant
    3bit-bits-in-byte: 37.9x as fast       325.6ms +/- 11.3%      8.6ms +/- 7.9%     significant
    bits-in-byte:      24.9x as fast       493.4ms +/- 8.0%     19.8ms +/- 19.6%     significant
    bitwise-and:       27.0x as fast       913.8ms +/- 13.5%     33.8ms +/- 9.5%     significant
    nsieve-bits:       9.40x as fast       536.0ms +/- 4.3%     57.0ms +/- 19.9%     significant

  controlflow:         78.6x as fast       440.4ms +/- 7.2%      5.6ms +/- 12.2%     significant
    recursive:         78.6x as fast       440.4ms +/- 7.2%      5.6ms +/- 12.2%     significant

  crypto:              10.8x as fast      1168.0ms +/- 5.7%    108.6ms +/- 12.5%     significant
    aes:               12.4x as fast       528.4ms +/- 6.3%     42.6ms +/- 11.4%     significant
    md5:               7.96x as fast       315.4ms +/- 8.8%     39.6ms +/- 33.3%     significant
    sha1:              12.3x as fast       324.2ms +/- 4.8%     26.4ms +/- 9.8%     significant

  date:                1.75x as fast      1620.4ms +/- 1.9%    928.4ms +/- 6.6%     significant
    format-tofte:      1.53x as fast       716.2ms +/- 3.1%    468.6ms +/- 6.0%     significant
    format-xparb:      1.97x as fast       904.2ms +/- 3.8%    459.8ms +/- 9.9%     significant

  math:                6.38x as fast      1524.8ms +/- 2.9%    239.0ms +/- 15.2%     significant
    cordic:            4.68x as fast       661.0ms +/- 2.9%    141.2ms +/- 27.4%     significant
    partial-sums:      5.74x as fast       426.0ms +/- 5.7%     74.2ms +/- 18.6%     significant
    spectral-norm:     18.6x as fast       437.8ms +/- 2.8%     23.6ms +/- 19.9%     significant

  regexp:              *1.06x as slow*     724.8ms +/- 3.0%    767.2ms +/- 3.3%     significant
    dna:               *1.06x as slow*     724.8ms +/- 3.0%    767.2ms +/- 3.3%     significant

  string:              4.57x as fast      5092.4ms +/- 2.0%   1113.6ms +/- 5.4%     significant
    base64:            16.0x as fast      2232.8ms +/- 4.5%    139.6ms +/- 20.7%     significant
    fasta:             8.52x as fast       994.8ms +/- 3.1%    116.8ms +/- 7.5%     significant
    tagcloud:          1.80x as fast       579.0ms +/- 4.3%    321.4ms +/- 8.5%     significant
    unpack-code:       1.78x as fast       698.2ms +/- 3.5%    391.2ms +/- 3.3%     significant
    validate-input:    4.06x as fast       587.6ms +/- 6.2%    144.6ms +/- 10.4%     significant

You can see that JavaScript performance is the major advantage to Chrome. They claimed in their comic intro that using multiple processes would result in less memory usage, but I didn't think so -- that woulden't make any sense. The figure next is a table from Google's own memory tool. I opened all of the same websites on Chrome and Firefox to see what the memory footprint really looked like. Keep in mind that Chrome base install is a very basic browser! My Firefox has a TON of useless plug-ins and themes installed and it still beat Chrome quite handily!

Memory (Google vs Firefox 3*)

 
About memory

Measuring memory usage in a multi-process browser

Summary

Memory
Virtual memory

Browser Private Shared Total Private Mapped

Chrome 0.2.149.27
154,844k 5,816k 160,660k 287,304k 107,336k

Firefox 3.0.1
132,572k 11,552k 144,124k 129,752k 11,844k

Processes

Memory Virtual memory

PID Name Private Shared Total Private Mapped

7172
Browser
37564k 21116k 58680k 43728k 26796k

832
Tab 2

iGoogle

Gmail - Inbox (9) - nzaugg@gmail.com
27616k 2660k 30276k 44416k 9816k

4456
Tab 3

CUEgle 3
8472k 2304k 10776k 32484k 9816k

3724
Plug-in

Shockwave Flash
56276k 10032k 66308k 94536k 11828k

7972
Tab 9

MSN Video
10872k 3376k 14248k 25140k 9816k

4348
Tab 10

Untitled
7736k 7380k 15116k 10396k 9816k

5552
Tab 14

Understanding User-Agent Strings
2668k 1736k 4404k 12256k 9816k

2420
Tab 17

Windows Live Hotmail
2304k 1624k 3928k 12540k 9816k

4204
Tab 20

SunSpider JavaScript Benchmark Results
1336k 2120k 3456k 11808k 9816k

5776
Tab 25 (diagnostics)

About Memory
8908k 3428k 12336k 9644k 9816k

Σ 163752k 219528k 296948k
117152k

	Memory	Virtual memory
Browser	Private	Shared	Total	Private	Mapped
Chrome 0.2.149.27	154,844k	5,816k	160,660k	287,304k	107,336k
Firefox 3.0.1	132,572k	11,552k	144,124k	129,752k	11,844k

		Memory	Virtual memory
PID	Name	Private	Shared	Total	Private	Mapped
7172	Browser	37564k	21116k	58680k	43728k	26796k
832	Tab 2 iGoogle Gmail - Inbox (9) - nzaugg@gmail.com	27616k	2660k	30276k	44416k	9816k
4456	Tab 3 CUEgle 3	8472k	2304k	10776k	32484k	9816k
3724	Plug-in Shockwave Flash	56276k	10032k	66308k	94536k	11828k
7972	Tab 9 MSN Video	10872k	3376k	14248k	25140k	9816k
4348	Tab 10 Untitled	7736k	7380k	15116k	10396k	9816k
5552	Tab 14 Understanding User-Agent Strings	2668k	1736k	4404k	12256k	9816k
2420	Tab 17 Windows Live Hotmail	2304k	1624k	3928k	12540k	9816k
4204	Tab 20 SunSpider JavaScript Benchmark Results	1336k	2120k	3456k	11808k	9816k
5776	Tab 25 (diagnostics) About Memory	8908k	3428k	12336k	9644k	9816k
	Σ	163752k		219528k	296948k	117152k

I haven't decided wither I will use this browser on a daily basis. Probably not -- no compelling reason to switch, but I can see that some will really like this browser. Perhaps once I get used to the idea of Google making a browser and have IE tick me off once more I'd be in a different place, but for now I kind of don't want a Google Browser.

Links

Chrome Download: http://www.google.com/chrome/

Chrome Comic Intro: http://www.google.com/googlebooks/chrome/

Chrome Features: (didn't work for me) http://www.google.com/chrome/intl/en/features.html

JavaScript Tests: http://17.254.17.57/perf/sunspider-0.9/sunspider-driver.html

Database Naming Conventions

Nathan Zaugg — Wed, 02 Jul 2008 20:50:00 GMT

Naming conventions are like arm pits. Everyone has them and they all stink! Well, at least that's the perspective of pretty much every developer an DBA alike. I will present my own personal philosophy for naming conventions on databases and hopefully spawn some discussion in the process.

Basic Principles

Consistency

As annoying as certain standards are (such as putting tbl_ before everything) it is more annoying and more difficult when there are no conventions or mixed conventions. Being able to reliably predict the schema once the basic relational structure is understood is key to productivity. Therefore, even if you get stuck with standards you disagree with, so long as they are consistent they will be much better than the alternative. Unless you get to make the decision, my guess is that there are going to be some conventions that you do not agree with.

Abbreviations

It is a good idea to abbreviate, when appropriate, in the naming of objects in your database. It may be a good idea to have a list of abbreviations that you plan to use in the database as part of your data dictionary. However, if there is not a good, clear abbreviation for an object, don't make one up. When in doubt, spell it out! Especially with SQL Server where you don't have the pesky 30 char limit for tables and columns like Oracle.

Identities

Every table should have an Identity as it's primary key! Sometime, in a future blog post I will explain why this is so critical, but suffice it to say that any table that does not have a primary key is considered by SQL Server a "heap". If you are using something other than an Identity column for the primary key you better have a really compelling reason because it will cause major performance problems. THERE IS NO SUCH THING AS A NATURAL KEY AND THEY SHOULD NEVER BE USED IN PLACE OF AN IDENTITY! So always use a surrogate key approach, even with join tables.

Security

I believe that with a good data layer like Linq to SQL there is no need for relegating all database access through stored procedures. While it does remove some of the service area for venerability and bugs robust solutions like Linq to SQL are very limited by this approach. You should grant specific access to tables and procs by user. A good approach can be found on another one of my blog posts.

Object Naming

Table Names

If you are running a database that preserves case (like SQL Server) tables should have no prefixes and should not contain underscores "_" unless it is a join table. Table names should also be Pascal Cased. If you are running a database that makes all tables upper case (like Oracle) then you have little choice but to use underscores everywhere.
Avoid pluralizing table names (User vs Users). This is a good idea for two reasons, first it can be confusing when doing the keys. Do we use UserID or UsersID? Second not all tables pluralize well (Addresses) so avoiding any plural names will keep it consistent. It you are using Linq to SQL the designer will pluralize for you automatically.
Join tables should the two or three tables that they are joining together as part of the name seperated by underscores. (ex: User_Address, User_Order). Although they are many-to-many relationship see if you can find a principle table. Users have orders, orders do not have users, therefore the User table comes first in the name.

Column Names

Name the Primary Key Identity column the table name with ID. (ex: UserID) With the possible exception of join tables, in which case just name the Identity ID.
Use Pascal casing (ex: EachWordsStartsCapolatized)
Do not use the table name as part of the column name. If this is a shipping table don't name your columns ShippingAddress, just name it Address.
Do not prefix column names with the type (ex: strUserName). It makes the database much more difficult to work with.
Use the correct data types. Always use nvarchar types (unicode) rather than varchar types. This avoids substantial complexity if you are ever requried to store non latin-based data! Trust me, you do not want to have to deal with code pages in the database! Also, use Date fields for dates, bit fields for boolean, etc.
Don't make every column nullable! Think through what data is absolutely required. If you want to hold "partally complete records" then I would suggest a different table or different "staging" database.
Don't make a bit field nullable unless you have a great reason!
Try to include a TimeStamp column if you think you may have to worry about concurrency.
Don't prefix with anything.

Constraint & Index Names

Name your constraints and indexes. With the exception of foreign key constraints they are not automatically assigned meaningful names.
Don't use prefixes and make light use of underscores.

Stored Procedure Names

Don't prefix your stored procedures! People used to prefix them with "sp" because existing procs in the database use this convention. It has been presumed that sp stands for system procedure and it wouldn't make any sense to use that. Seriously, prefixes are not very helpful in the database!
The first part of the name of a proc should be the table name it works upon (ex: User_Insert). If the proc works on multiple tables try to give it the name of the portion of the database this proc deals with. For example, if it's a proc that the invoicing system uses it would be acceptable to name it Invoicing_Update, for example.
Don't generate procs for simple Insert, Update, Delete, and Select unless you have a policy in place for accessing data exclusively from procs.
Don't create any stored procedure you don't need or plan to immediately use. At some point you will change the schema and you won't update procs your not using. Someone may eventually want to use that proc later only to find it broken.
The verb in the naming convention does not have to be relegated to "Insert, Update, Delete, Select". It should say what it does. Just be careful that if there is another procedure that does this same thing to another table that the verbs are named the same.
You can add additional information to the proc name to help distinguish it from others. (ex: User_Select_ByDate, User_Select_ByState)
Don't use a prefix for arguments (ex: @ArgUserID). In my experience they don't help at all and are quite annoying!

Tips & Tricks

SQL Server 2008 has a policy manager that can help create and enforce policies like naming conventions! Regardless of using SQL 2008 be sure to keep a Data Dictionary of your database! The database is the heart and soul of your business processes and should be well documented! There is nothing worse than an unclean database!

Nathan Zaugg

Avoiding Circular Dependencies

Nathan Zaugg — Mon, 09 Jun 2008 21:45:06 GMT

The term "circular dependency" may be foreign to some programmers (especially if you do Java as it is a pretty common practice). However, anyone who has done some scripting for a referential database knows that you have to run scripts in a certain order. Running scripts out of order causes errors when you run. The interesting trick is that if you run that same incorrectly-ordered DDL script again and again you will eventually get it to run without errors. If you were unaware of the order being incorrect and thought to yourself in that moment "Stupid database!" then this blog post is for you!

What is a circular dependency?

It is simply two libraries that use each other (either directly or indirectly) as shown below:

Figure 1: Circular Dependency

Figure 2: Complex Circular Dependency

The complexity of a circular dependency may vary. If you are using Visual Studio and have all of your projects loaded into a single solution AND you add Project References (Right click on project -> Add Reference -> Projects Tab -> {Project Name}) then the IDE will not allow you to create Circular Dependencies. In fact, this is a good practice as Visual Studio will ensure the correct build order.

Why are circular dependencies bad?

Just like our Database example above, a circular dependency makes it so you can not guarantee that your application has the latest code. That is a big deal! Here is why:

I make changes to Application 1 (in Figure 2)
I build my project, The changes I made in Application 1 may or may not have gotten into Application 2 (depending on build order). It may have taken a copy of the compiled code that was left over from the last time I built.
Application 2 depends on this new functionality to provide services to Application 3; This functionality will not work correctly with this build.
Application 3 may or may not depend on these same services to provide back to Application 1

As you can see in this scenario, there is no such thing as a "correct" build order when there are these circular dependencies. The only way you can arrive at the correct version of the code is to build it as many times as there are nodes in our circle. That would mean for Figure 1 that we would need to build twice and three times for Figure 3. Some of these dependencies can get really ugly! Here is some actual code running in an actual company that I did analysis on some time ago using a tool called Structure 101.

How do I fix circular dependencies?

There are some steps to take to solve even the most complex tangles! They all involve refactoring your code though.

Refactor common code into a "base" dependency; I usually call this "Common" (figure 3). BEST SOLUTION
Remove code that is unused. In the tangles shown above many of them are using deprecated/unused code.
Duplicate the sections of code used. This should be seen as a last resort but given the choice between code duplication and circular dependencies, I take code duplication ever time!

Figure 3: Refactor a Common

Summary

There are two kinds of design concepts for nTier (and other types of architectures as well) called Logical Layout Design and Physical Layout Design. The Logical Layout is simply that your software occupies the same project/package but leverage different classes. In contrast Physical Layout Design forces each tier to be separated into different Projects/Packages. So long as we are careful to manage the dependencies between these packages from the start this is the preferable way to code. While the logical layout does not suffer from the dependency problem eventually you may wish to break these classes apart and find that there are a lot of inner-dependency that should not exist simply because they occupied the same project. Remember to keep it clean!

Unit Testing Philosophy -- Code Coverage is meaningless....

Nathan Zaugg — Wed, 23 Apr 2008 22:38:17 GMT

While I am a big fan of unit testing I often try to point out that Code Coverage tells us little more than “are there unit tests” not “are we unit testing”. The former indicates that the code is indeed being executed. The latter indicates that the code is being exercised.

Scott Hanselman describes this in a podcast available here:
http://www.hanselman.com/blog/HanselminutesPodcast103QuetzalBradleyOnTestingAfterUnitTestsAndTheMythOfCodeCoverage.aspx

I personally think testing really needs to be evolved to the level our architecture and code. Testing frameworks are clumsy and it is very difficult to all of our code. Most people are quick to point out that unit testing UI is difficult and usually yields the least amount of benefit. Depending on the architecture and implementation of that specific application, that may well be true.

Two other major hurtles that are well known. First, when we unit test we want as “authentic” of a test as we can get. This includes the data. How much time and effort do we spend getting authentic data to our unit tests or providing an abstracted data layer for our code to operate on? This is a big problem not just because of the hours it takes to provide such data abstraction implementations but because the time it takes to run Unit Tests is greatly increased. I argue this because I believe that when unit testing takes 30 min’s to complete, developers are not going to run them before checking in, and that I believe is an issue.

The other major unit testing hurdle is the concept of the “Oracle” as described by Scott. This term simply refers to the ability of a piece of code to assert that a test has indeed run in such a way that an end user is going to get what they expect. Some of these sceneries can be very complex and it is impossible to test for all permutations of a given rule.

What do I think is needed in Unit Testing?

The disparity between code and “UI” needs to be removed! It needs to be no harder to test a piece of UI then a API Class Library.
Unit testing “language” needs to be more declarative and functional. I need to be able to test a wide variety of test scenarios quickly and easily. Functional languages are good at this. I do not suggest that we adopt Python for unit testing but maybe a testing language developed for this purpose.
There needs to be a healthy attitude toward testing in the industry. I watch every day at any place I go where people “fix” the code in production. If they had better testing before the code went to production they could have easily caught these errors! A production line is down for an entire day, but if you ask -- no one can afford
There needs to be more automation in Unit Testing. For example, can’t the computer automatically generate tests for the edge cases? Coulden’t it generate input that would “execute” every branch of code possible? It may be of marginal usefulness but it would be free.
Unit testing needs to run faster! I have long mentioned the need to use multiple threads to execute tests – I even modified a release of nUnit to run on the ThreadPoool, but these need to be adopted and standardized
There needs to be a better way to test with real data.

Of course, the best sets of testing will not eliminate all bugs. In fact, you must adopt your own testing philosophy when in charge of a development effort. The question you have to ask yourself is two-fold. First, How important is it that my code has very few bugs. Would a bug in your software, if unchecked, cost your company a lot of money? Could a potential bug cost the company thousands, millions, billions, or even someone's life? If so, then your testing should reflect that. It's good risk management to spend 2x or more the cost of software development to avoid this type of costly loss. On the other hand, if software has little consequence other than user annoyance when a bug is present then perhaps less of the development dollars need go toward testing. In some sceneries it is completely fathomable that none of the software budget is spent on automated testing.

The other question you will need to answer is will adding automated testing to my project yield a return on investment? Before you make a snap judgement on this second point I may add that systems that have good testing in place (automated testing in particular) have a much cheaper lifecycle than applications that do not have this type of safe-guard. Automated testing is a lot like life insurance in this way. Having critical pieces of your code unit tested will save your bacon almost guaranteed!

Nate's Stuff : Architecture

Visual NDepend Blows My Socks Off!

Installation Experience

Analyze a Project

Results

Links:

The Service Oriented Architecture (SOA) Philosophy

Background

What is a service?

Figure 1: Enterprise SOA Diagram

Why SOA?

Testability

Security

Simplicity

Performance

Configuration Management / Distribution

Summary

Google releases web browser "chrome"

Install

What Google did right

What Google didn't do right

Google vs IE 8 (in IE 7 Emulation Mode)

Google vs Firefox 3

Google vs IE 8

Memory (Google vs Firefox 3*)

About memory

Summary

Processes

Links

Database Naming Conventions

Basic Principles

Consistency

Abbreviations

Identities

Security

Object Naming

Table Names

Column Names

Constraint & Index Names

Stored Procedure Names

Tips & Tricks

Avoiding Circular Dependencies

What is a circular dependency?

Figure 1: Circular Dependency

Figure 2: Complex Circular Dependency

Why are circular dependencies bad?

How do I fix circular dependencies?

Figure 3: Refactor a Common

Summary

Unit Testing Philosophy -- Code Coverage is meaningless....