“Intellipad” is a .NET application that is used to write a .NET application: (large portions of) itself.

Most of the commands are in IronPython.  Further, the tests are in IronPython.  You can run them from the command buffer in the tool (>>>Test(“TestName”)) and they ship in the CTP.

Beyond IronPython, “Intellipad” supports “modes” for “M”, T-SQL, and lots more.  Of course, since the tool is completely extensible, you can add whatever language you want, as I did with MUrl.

BTW:  In case it is not clear, this is all still in CTP, the team is still working on it, so stuff can change.  I know the “M” and “Intellipad”/”Quadrant” teams believe strongly in text as a core way of “modeling” software, so you can expect that principle to be honored.

I often get asked about this project and it is a recurring search leading to this blog, so I thought I would give an update on what happened since we staffed up that team.

The short story is that we gave this project an official unofficial name: “Intellipad”.  This tool was included in the first “Oslo” CTP and continues to ship in the latest SQL Server Modeling CTP.

We designed “Intellipad” to work standalone as a lightweight companion to “Quadrant” and VS.  In addition, a core scenario was in situ use within “Quadrant” itself.

You can see three “Intellipad” views in “Quadrant” below (from the current CTP).  One shows “M”.  Another shows the T-SQL generated from the “M”.  The last shows “Intellipad” as the query editor in the table view.

The ‘Quadrant” integration is something we debuted at PDC this year, but the fact that we were going there shouldn’t have been a surprise.  First, I told anyone that asked and second the name is the best clue.  “Intellipad” is short for “Intellisense Workpad”.  A workpad is the name that we give for a view inside of “Quadrant”.

BTW:  One of the main developers of “Intellipad” and “Quadrant” has an impressive side-project that you may want to check out: SmallBasic.

[Update:  If you want to see what you can do with “Intellipad”, check out how we extend it for MUrl.]

I enjoyed doing the podcast.  Keith/Woody were great hosts.

You can listening at http://tinyurl.com/deepfried43.

.

Billy Newport, an IBM Distinguished Engineer, was interviewed recently on why they selected a RESTful data service as the API and how OData helped.

The article: http://searchsoa.techtarget.com/news/article/0,289142,sid26_gci1379765,00.html

More product details: http://www.ibm.com/developerworks/websphere/downloads/xs_rest_service.html 

It is great to see that developers, regardless of platform/language, have a simple way to consume these services.

I’ll make one interesting note about this implementation.

As near as I can tell, Billy’s team implemented OData without ever talking to anyone at Microsoft.

I suspect they used the protocol documents we have online (these define the protocol with even greater precision that many standard specifications I have seen) and a HTTP trace tool.

Having been involved in distributed computing/protocol integration work for a long time, that is quite an achievement.

It could speak to simplicity of the protocol (it is just conventions/extensions over HTTP/AtomPub), the quality of the documentation or the intelligence/patience of the IBM team.

Likely it was all of these.

Back in the “D” days, our team started working with MSR on a number things related to the language.

We recently released some of that work on the MSR site: Dminor.

Dminor is a data-modeling language based on M, the data-modeling language of Microsoft Oslo. Dminor provides extended compile-time checking of code, making use of an SMT (Satisfiability Modulo Theories) solver.

You can read more about it at http://whigmaleerie.spaces.live.com/blog/cns!C6149B019D236BF5!846.entry.

You can download it at http://research.microsoft.com/en-us/downloads/cd60cdb0-353f-48b3-81d7-177621eba1bf/default.aspx.

Congrats to Andy and company…

One of the major new WCF features in .NET 4 is the Routing Service which is a configurable WCF-based service that supports content-based routing and protocol bridging. The content based routing capability in .NET 4 allows for WCF to perform message filtering based on content contained in either the SOAP headers or within the message body. For instance, if a company has two different versions of the same service deployed, the client application can call into a central routing service. The routing service can then forward the message to the correct backend service based on information it extracts from the incoming message, such as version number. The routing service also supports error handling routing that can automatically resend the message to another destination endpoint in the event of an error.

Unfortunately, the default routing service is not the easiest to manage or configure. Fortunately, the recently released SQL Server Modeling CTP makes it easy to build model driven application and the .NET 4 router configuration is a good modeling candidate. As such, the RouterManager sample which is available for download from MSDN Code Gallery, shows how this can be accomplished using Visual Studio 2010 Beta 2 and the SQL Server Modeling CTP which was released in conjunction with PDC09. This is the first in a series of articles that will cover the architecture and concepts used in the RouterManager sample application to create a fully model driven content based router.

The SQL Server Modeling CTP can be used to create a domain specific language (DSL) that can make the routing configuration of the WCF routing service much simpler. This is accomplished by first defining a language that is human readable, which will provide IT Operations and others within an organization who are not expert developers with an easier way to interact with applications. The CTP’s modeling language features make it easy to define the structure of the configuration data in a human readable representation that can be shared within an application’s modules, tiers and tools or between even between applications. Also included, as part of the CTP, is the SQL Server Modeling Services, which provides functionality to store, access and share both the physical and metadata representation of the model. The Modeling CTP also provides a visual tool called “Quadrant” for viewing and interacting with models and model data.

http://blogs.msdn.com/dmcat/archive/2009/11/16/model-driven-content-based-routing-using-sql-server-modeling-ctp-part-i.aspx

http://www.sixpairs.com/intellipad/mgraph-visualization-intellipad-plugin.aspx

There is a good readme in the zip file that outlines how to build, install and use it.

Download it now. 

[Updated on 2010/1/4 to fix a bug in installation]

Twitter.murl

How the credential information is stored

@douglasp In .edmx I could add attributes in my own namespace to enrich my model. How is that going to work with ‘M’?

In the latest CTP you can add attributes/annotations for “M” types, extents, fields, functions.

Below is a simple example.  Note the SQL that is generated by default, but you can control that as well.

The below screencast runs ~2 minutes and worth watching if you care about interacting with data (and who doesn’t).

What you see are 3 key features of “Quadrant” in the latest CTP:

• Access to SQL Azure databases
• Creating custom views (including master detail) by mashing views together
• See the “markup” of any view in Quadrant and steal it/change it (just like view source in a Web browser)

You also get to see the infinite canvas and the zoom features indirectly.

I am thinking about posting a couple more of these 2 minute demos over the course of the next couple weeks.

If you want to see something more in-depth now, check out the “Quadrant” PDC talk or Quadrant Overview video.

[Updated:  Download Video @ http://www.douglaspurdy.com/wp-content/uploads/quadrant1.swf]

I could spend a bunch of time talking about the design principles behind that work, but I want to be as concrete as I can and get down to something that help you (if you are a .NET developer) today.

In a nutshell, “M” replaces the EDMX XML file that you are no doubt familiar with if you have used either the Entity Framework or Data Services.

Said differently, we have a nice human-friendly DSL that replaces a human-hostile XML file.

In addition, this nice, warm and friendly DSL can both generate your database and the EF classes you need to program against the database.

Before I talk about how this works, I want to say one thing:

EF and DS are at the knee of the adoption curve and we are not going to do anything to slow down that adoption; so rest assured that we are committed bring EDM and “M” together in a way that places the upmost importance on existing and future investments in EF and DS.

With that said, let’s look at a simple example (if you want to follow along, download the bits first)…

1. Look at all the “Oslo” VS projects that we have in the CTP bits (the name landed too late to get the bits updated).

2.  Let’s create a simple “M” model

3. Set the connection string in the project properties

4. Deploy (this will create the database and the CLR classes you need)

5. Write some code against the generated classes

6. Run the App

8. Under the covers (you can look in the obj folder to see what we generate)

9. Really under the covers, especially if you don’t like to use an IDE (you know who you are)

This uses a command-line option for m.exe and tool that will generate the EF classes for you.

I am fairly sure that someone on the language team is going to throw something at me when they read this.

When Sells and I talked about this feature at PDC, it resulted in at least two separate rounds of applause from the audience and I know it was a highlight (one of many) at the Box/Anderson anchor session.

You can get more details on this at http://msdn.microsoft.com/en-us/library/ee713219(VS.85).aspx.

This is one of the best articles that I have read about the Microsoft data and modeling history and strategy. 

Kraig wrote this piece and I love it.

Below is at the end of the doc that talks about modeling…

Data Development Futures: Modeling

If we take a step back from the whole arc of the discussion we’ve had in this article, we can see a definite trend toward greater levels of abstraction. The earliest data access solutions for SQL Server, like DB-Library and ESQL for C, were sufficient though quite rudimentary. Technologies like ODBC created an abstraction layer above the proprietary APIs of different databases, a model that OLE DB, ADO, and ADO.NET continue to follow.

Most recently, the Entity Framework has gone a step further to create an additional abstraction layer not over the data access API but over the structure of a relational database itself. Similarly, Data Services transform any number of diverse data sources into something accessible through a simple REST-based exchange protocol. (In fact, Microsoft expects that using such protocols will become increasingly popular, as it allows data providers and consumers to evolve independently from their programming model.)

Taken as a whole, this trend can be described as a trend toward modeling, the act of creating representations of real-world concepts that are internally translated into the representations that computer systems (like database engines) inherently understand.

What’s key here is that modeling itself is something that takes place outside of the database, in large part because data often spans multiple databases and multiple formats. Thus what matters is the richness of the pipeline between what’s in the data store and what runs outside that store.

Increasing that richness is the purpose of Microsoft’s next wave of investments beyond the Entity Framework and Data Services (see Entity Framework futures and Data Services futures), namely a body of technologies collectively called the SQL Server Modeling CTP (Community Technology Preview). Rather than replacing existing data development methods, these technologies introduce new ways of working with SQL Server databases along with greater availability of metadata alongside the data itself.

Metadata—or information about data and applications—is the key to the next advances in developer productivity with data-oriented applications. Modeling, in other words, is a recognition that data about an application, and data about data, is just as important as the application and data themselves. And the components of the SQL Server Modeling CTP, shown in Figure 10 as a projection from Figure 9, are the ways in which Microsoft is beginning to explore this new territory.

Figure 10: Future technologies in the SQL Server Modeling CTP.

The code name “M” language is like a more manageable (though more limited) form of Transact-SQL, the language normally used to describe data schema and values. It’s also closely aligned with the purpose of the EDMX files used in the Entity Framework to implement the conceptual layer (or Entity Data Model). The "M" language is, in fact, being developed as a textual language that also implements the Entity Data Model, and will serve alongside the XML-based EDMX dialect. Furthermore, "M" includes powerful support for the creation of domain-specific languages (DSLs) in which application logic, data access layers, and even an application’s user interface can be easily defined by even non-developers.

SQL Server Modeling Services, for its part, deals with enterprise-wide metadata and the kinds of applications that enterprises are interested in building around that metadata. And the code name “Quadrant” tool provides a visual means of interacting with relational data in ways that have traditionally been either very difficult or have necessitated a custom application of some kind. Many “forms over data” applications that have to date been written using the other data access technologies we’ve seen can be quickly assembled directly within “Quadrant”.

If you have been skeptical of our recent renaming of “Oslo”, I encourage you to actually watch the videos, download the bits and then make up your mind.

More importantly, if you are a .NET developer, please take a look at what we are doing around EF and DS – especially the OData protocol.

Data/Modeling “Anchor Session”

Data Programming and Modeling for the Microsoft .NET Developer

Coming in VS 2010

Evolving ADO.NET Entity Framework in .NET 4 and Beyond

ADO.NET Data Services: What’s new with the RESTful data services framework

Developing REST Applications with the .NET Framework

SQL Server Modeling (nee “Oslo”)

Microsoft Project Code Name “M”: The Data and Modeling Language

Building Data-Driven Applications Using Microsoft Project Code Name "Quadrant" and Microsoft Project Code Name "M"

SQL Server Modeling Services: Using Metadata to Drive Application Design, Development and Management

Using SQL Server Modeling

The ‘M’-Based System.Identity Model for Accessing Directory Services

This name continues our tradition of naming functional components beyond the core engine with the services moniker.

Other such services are Reporting Services, Analysis Services, Integration Services, etc.

Shosh will be talking more about it at PDC at SQL Server Modeling Services: Using Metadata to Drive Application Design, Development and Management.

We are still working on the official names for “Quadrant” and “M”.  We have some good candidates, but we didn’t get to close for PDC.

Just as a reminder:

“Oslo” –> SQL Service Modeling

The components of the SQL Server Modeling are:

• “M” is a highly productive, developer friendly, textual language for defining schemas, queries, values, functions and DSLs for data
• “Quadrant” is a customizable tool for interacting with large datasets stored in SQL Server databases
• SQL Service Modeling Services (nee “Repository”) is a SQL Server role for the the secure sharing of models between applications and systems

System.Identity is built by the Active Directory team.

It is one of the first uses of SQL Server Modeling (nee “Oslo’) that we are talking about publically (you will see others, but this is the first announced Microsoft project that is ready to have a PDC session).

If you are interested in how a real and big ISV (they are considered an ISV for us) uses SQL Server Modeling, this session should be of interest to you.

The ‘M’-Based System.Identity Model for Accessing Directory Services

First, the “M” language is not tied to any Microsoft implementation.  Not SQL Server.  Not Windows.  Not anything.

We have committed to publish the “M” language specification under the OSP.  There is already a JavaScript implementation.  You are free to write your own.

Second, the Microsoft implementation of the “M” language will ship with SQL Server.  It could also ship with a number of other Microsoft products, but of all the things in this uncertain world, the most certain thing I know is that “M” is going to ship with a future version of SQL Server.

BTW: When I mean SQL Server, I do not mean the engine.  I mean the broader SQL Platform that you may have heard called the Data Platform or Information Platform.  SQL Server Express is part of that.  SQL Azure is part of that.  Note that we don’t charge a dime for SQL Server Express.  We have not made any decisions about how we are going to license our implementation, but I will tell you – I know that history is kind to technologies that are broadly available to developers.

Let’s recap.  “M” is an open language that anyone can implement.  Our implementation of that language will at least ship in SQL Server (the broader definition) and be available in products that use SQL Server (the broader definition).

BTW: Did you know that SQL Server Express is available for free, right inside the Web Platform Installer.  Easy to install.  Easy to use.

Lastly, as I have stated in lots of comments and posts, the core DSL capabilities we have in “M” remain.  Sells just posted an entry that shows more of the new capabilities available in the CTP.

One thing I am committed to doing is answering every single comment, tweet and email that I can until I think we have been absolutely clear on what we are doing with SQL Server Modeling and “M” in particular.

So if you still have questions, keep them coming…

Some comments were positive, others helpful, others disappointing and others a little beyond the pale, particularly ad hominem comments (for the record, I remember the fall of the Berlin Wall very well).

Regardless of the message, we appreciate the feedback, good or bad.

Broadly, many of the comments that I would classify as “disappointed” seem to break down into two categories:

1. How can this possibly be part of SQL Server?  I thought “Oslo” was a modeling platform for developers.  Aren’t developer things called Visual Studio and .NET?  This must be a complete failure if it is not part of VS and .NET.
2. What about the DSLs story?  I am confused how a DSL story can come out of something called SQL Server.  Shouldn’t this be part of .NET?  They must have abandoned the DSL story completely if it is not part of .NET.

Let me address each of the above in turn…

As for #1, this is going to be more involved than I typically do in a blog, but I think it is important for developers to understand.  This could be remedial for some; my apologies in advance.

Microsoft has a number of different products including Windows (Client and Server), Office and SQL Server.  Visual Studio is a developer tool that developers use to target these products.  Both we and the industry offer more than just this tool.  The .NET Framework is the name of one of the frameworks that developers can use to target these products.  Both we and the industry offer more than just this framework.  When we ship a new version of one of the above products, we ship programmability features that allow developers to use the new features in the product.  More times than not, these programmability features show up in VS and .NET.

Let me make this concrete.  In SQL Server 2008 R2 we have new feature called DAC.  DACs are just like a MSI or MSDeploy package, but for the database.  This is the way that we want developers to build SQL Server databases moving forward, as it is provides a model for the DDL/DML that we can inspect and offer services over.

We ship a VS project for creating DACs (watch the video here).  It offers a very compelling developer experience.  In fact, one developer inside of Microsoft said the other day to me out of the blue (we are working on a PDC demo) [Updated:  This was Jonathan Carter.  I just ran into him backstage at PDC and told him I would attribute.]:

I spent some time this evening digging into DAC projects, and I’m realizing just how awesome they are. … This is very compelling.

In addition to the VS tooling, we also ship a .NET assembly that lets a developer operate on top of a DAC package and do their own interesting things.

The key takeaway is that this is a SQL Server product feature that has programmability via .NET and VS.  It is just as much a part of VS and .NET as anything else we ship.

As a developer, if you look at all the things you program against in VS/.NET you will quick see that most of what you are programming are actually features of the underlying products like Windows, Office and SQL Server.

Just to drive this point home (and make another one), there are a countless SQL Server features that manifest in VS/.NET that target other databases than SQL Server.

Look at Reporting Services.  Look at Analysis Services.  Look at Integration Services.  All have programmability via VS/.NET and they let developers interact in a rich way with data in Oracle, etc.  Perhaps we will see something like Modeling Services in the near future…

In terms of why SQL Server is the right home for this set of technologies (rather than the other products like Windows and Office), I would hope that would be obvious based on my previous post, but I’ll provide a simple summary:  Models are data.  It seems logical that our modeling technologies would be a part of our data/information platform offering.

As for #2, that is very straightforward.  The core DSL capabilities we have in “M” remain and we expect those capabilities to be part of the language when we ship.  We think that DSLs, both visual and textual, are important aspects of a holistic modeling platform.  I’ll hedge a little (based on previous experience) and say that we still aren’t done, but we do have a ship vehicle (with a hard schedule), so the likelihood of change is much smaller than anything we had on the project previously.

While that modeling platform is now part of SQL Server, that does not mean that there is not a VS or .NET experience for this aspect of the language.  If you recall, SQL Server already supports many languages (including .NET languages that run both in the store and outside).

As evidence of these capabilities, let me show you a screenshot of something we included in the PDC CTP that the languages team was happy to get in (Pinky and Don are likely to hit me with something tomorrow for showing this):

The net of this post is the following:

• The fact that these technologies are part of SQL Server does not mean that they are not available in Visual Studio or part of the .NET Framework – they are absolutely deeply integrated with both VS and .NET.
• SQL Server, of all the Microsoft products, is the most obvious and logical place for these technologies to be located.
• We remain committed to the core DSL capabilities of the “M” language.

If you still have questions about what we are doing, I encourage you to download the SQL Server Modeling CTP when we release it at PDC from http://msdn.microsoft.com/data and watch the PDC sessions online when they are posted next week.

[Update:  I address more “M” specific questions:  On “M”]

• Microsoft’s “Oslo” Becomes SQL Server Modeling
• Microsoft whittles away at Oslo; now plans to fold it into SQL Server
• [Updated] Microsoft rechristens Oslo modeling platform
• [Updated] Remodeling Oslo
• [Updated] Microsoft rechristens Oslo modeling platform
• Twitter

First, based on conversations that I am having on email and Twitter, one of the things we need to do is explain how we think the DSL capabilities of “M” manifest in SQL Server.  My previous post was all about the macro move of “Oslo”, not specific features.

I’ll post something on that topic this week (PDC is pounding me into fine dust right now).

Second, it is important to bear in mind that SQL Server is a big tent, likely bigger than you have thought, including things like operating over heterogeneous data, Office integration, etc.

Third, putting these technologies in SQL Server makes it more likely for any aspirations we had to come to fruition; as almost every single enterprise product that Microsoft produces leverages SQL Server – and this adoption is only getting greater.

Lastly, I think the PDC bits and sessions are going to make it very clear how this is going to help mainstream developers be successful and more productive writing applications.

Watch the sessions, use the bits, and decide.

In the year since PDC 2008, we delivered three public CTPs and conducted many software design reviews (SDRs) with key customers, partners and analysts.  We listened intently to the feedback and it helped us to shape our approach toward bring this technology to market.  With PDC now one week away, we are beginning to disclose the next chapter in the journey to “Oslo”, with more to be unveiled at various keynotes and sessions at the PDC event itself.

Of the key things we observed over the last year was the real, tangible customer value in applying “Oslo” to working with SQL Server.  Time after time we heard that “M” would make interacting with the database easier, provided we offered a good end to end experience with tools (VS) and frameworks (Entity Framework and Data Services) that developers use today.  We heard that developers wanted to use the novel data navigation/editing approach offered by “Quadrant” to access their data in whatever SQL Server they wanted, not just the “Repository”.  We heard that the notion of a “Repository” as something other than SQL Server was getting in the way of our conversations with customers.

Another thing we learned was that most of the customers that we wanted to leverage the modeling platform were already using SQL Server as their “repository”.  Take an application like SharePoint.  It is already model-driven.  It already stores its application definition in a database.  Dynamics is the same way.  Windows Azure is the same way.  System Center is the same way.  What we didn’t have was a common language, tools or models that spanned all of these applications, although they were all leveraging the same database runtime.  The simplest path to get all of these customers sharing a common modeling platform seemed obvious.

Lastly, we learned that the folks on the SQL Server team were hearing the need for additional mechanisms to make the database more approachable to developers.  Developers did not want use three different languages to build their database applications (T-SQL, a .NET language and a XML mapping file).  Developers wanted new tools that let them deal with the truly massive amount of data they need to handle on a daily basis.  Developers wanted to radically simplify their interactions with the database, with a straightforward way of writing down data and getting an application as quickly as possible.

With all of the above in mind, we just announced (at VS Connections) the transition from “Oslo” to SQL Server Modeling.  At PDC, we will release a new CTP using this name, SQL Server Modeling CTP, that will begin to demonstrate how developers will use these technologies in concert with things like T-SQL, ADO.NET, ASP.NET and other parts of the .NET Framework to build database applications.

The components of the SQL Server Modeling CTP are:

• “M” is a highly productive, developer friendly, textual language for defining schemas, queries, values, functions and DSLs
• “Quadrant” is a customizable tool for interacting with large datasets stored in SQL Server databases
• “Repository” is a SQL Server role for the the secure sharing of models between applications and systems

We will announce the official names for these components as we land them, but the key thing is that all of these components are now part of SQL Server and will ship with a future release of that product.

At PDC, we will unify the “Oslo” Developer Center and the Data Developer Center.  You will be able to find the new SQL Server Modeling CTP at our new home (http://msdn.microsoft.com/data) the first day of PDC.  I encourage you to download this CTP and send us your feedback.

If you are attending PDC, we have some great sessions and keynotes that will highlight the work we are doing with SQL Server Modeling.  My personal favorite is “Active Directory on SQL Server Modeling” (the actual title is The ‘M’-Based System.Identity Model for Accessing Directory Services), which is going to show how a serious “ISV” is using these technologies.

Speaking for myself and the team, we are very excited about this transition.  Many of us have worked on numerous “v1” products while at Microsoft.  This sort of transition is exactly what successful “v1” products/technologies undergo based our collective experience.  You have a vision based on customer need.  You write some code.  You get customer feedback.  You adjust.  You repeat.  You find the place that maximizes your investment for customers.  You focus like a laser on delivering that customer value.  You ship.

Looking forward to the next chapter…

[Updated: follow-up to this post:  On DSLs and a few other things...]

[Updated: another follow-up post: On “M”]

[Updated: Last follow-up post with pointers to PDC videos]

Shosh tells me that this event is valuable for both woman and men in the technology field.

I plan on attending.

It is on the 18th — no time is on the event page– but I using my super high IQ, I have to assume that it is some time around lunch.