On DSLs and a few other things…
My initial post on the retiring of the “Oslo” project name drew a lot of attention and comments.
Some comments were positive, others helpful, others disappointing and others a little beyond the pale, particularly ad hominem comments (for the record, I remember the fall of the Berlin Wall very well).
Regardless of the message, we appreciate the feedback, good or bad.
Broadly, many of the comments that I would classify as “disappointed” seem to break down into two categories:
- How can this possibly be part of SQL Server? I thought “Oslo” was a modeling platform for developers. Aren’t developer things called Visual Studio and .NET? This must be a complete failure if it is not part of VS and .NET.
- What about the DSLs story? I am confused how a DSL story can come out of something called SQL Server. Shouldn’t this be part of .NET? They must have abandoned the DSL story completely if it is not part of .NET.
Let me address each of the above in turn…
As for #1, this is going to be more involved than I typically do in a blog, but I think it is important for developers to understand. This could be remedial for some; my apologies in advance.
Microsoft has a number of different products including Windows (Client and Server), Office and SQL Server. Visual Studio is a developer tool that developers use to target these products. Both we and the industry offer more than just this tool. The .NET Framework is the name of one of the frameworks that developers can use to target these products. Both we and the industry offer more than just this framework. When we ship a new version of one of the above products, we ship programmability features that allow developers to use the new features in the product. More times than not, these programmability features show up in VS and .NET.
Let me make this concrete. In SQL Server 2008 R2 we have new feature called DAC. DACs are just like a MSI or MSDeploy package, but for the database. This is the way that we want developers to build SQL Server databases moving forward, as it is provides a model for the DDL/DML that we can inspect and offer services over.
We ship a VS project for creating DACs (watch the video here). It offers a very compelling developer experience. In fact, one developer inside of Microsoft said the other day to me out of the blue (we are working on a PDC demo) [Updated: This was Jonathan Carter. I just ran into him backstage at PDC and told him I would attribute.]:
I spent some time this evening digging into DAC projects, and I’m realizing just how awesome they are. … This is very compelling.
In addition to the VS tooling, we also ship a .NET assembly that lets a developer operate on top of a DAC package and do their own interesting things.
The key takeaway is that this is a SQL Server product feature that has programmability via .NET and VS. It is just as much a part of VS and .NET as anything else we ship.
As a developer, if you look at all the things you program against in VS/.NET you will quick see that most of what you are programming are actually features of the underlying products like Windows, Office and SQL Server.
Just to drive this point home (and make another one), there are a countless SQL Server features that manifest in VS/.NET that target other databases than SQL Server.
Look at Reporting Services. Look at Analysis Services. Look at Integration Services. All have programmability via VS/.NET and they let developers interact in a rich way with data in Oracle, etc. Perhaps we will see something like Modeling Services in the near future…
In terms of why SQL Server is the right home for this set of technologies (rather than the other products like Windows and Office), I would hope that would be obvious based on my previous post, but I’ll provide a simple summary: Models are data. It seems logical that our modeling technologies would be a part of our data/information platform offering.
As for #2, that is very straightforward. The core DSL capabilities we have in “M” remain and we expect those capabilities to be part of the language when we ship. We think that DSLs, both visual and textual, are important aspects of a holistic modeling platform. I’ll hedge a little (based on previous experience) and say that we still aren’t done, but we do have a ship vehicle (with a hard schedule), so the likelihood of change is much smaller than anything we had on the project previously.
While that modeling platform is now part of SQL Server, that does not mean that there is not a VS or .NET experience for this aspect of the language. If you recall, SQL Server already supports many languages (including .NET languages that run both in the store and outside).
As evidence of these capabilities, let me show you a screenshot of something we included in the PDC CTP that the languages team was happy to get in (Pinky and Don are likely to hit me with something tomorrow for showing this):
The net of this post is the following:
- The fact that these technologies are part of SQL Server does not mean that they are not available in Visual Studio or part of the .NET Framework – they are absolutely deeply integrated with both VS and .NET.
- SQL Server, of all the Microsoft products, is the most obvious and logical place for these technologies to be located.
- We remain committed to the core DSL capabilities of the “M” language.
If you still have questions about what we are doing, I encourage you to download the SQL Server Modeling CTP when we release it at PDC from http://msdn.microsoft.com/data and watch the PDC sessions online when they are posted next week.
[Update: I address more “M” specific questions: On “M”]
[...] [Updated: follow-up to this post: On DSLs and a few other things...] [...]
From “Oslo” to SQL Server Modeling at Douglas Purdy
12 Nov 09 at 06:51
Doug, just a question. Did you see Sculpture (http://www.dawliasoft.com/Home/tabid/37/Default.aspx)?
What do you think about such approach to data programming?
j.
Jacob
12 Nov 09 at 08:52
I think it might be worth describing concrete examples of the types of problems you expect Oslo to solve and how it solves them.
Colin Jack
12 Nov 09 at 09:20
Text DSL to describe database is great. The main problem I see is migrations. What to do when column renamed? In DataDude this is already addressed, by refactoring log. In ruby by migrations. I am waiting for PDC videos to get questions answered.
Mike Chaliy
12 Nov 09 at 11:45
@Mike Chaliy I completely agree. We have spent a bunch of time on this problem over the past few months. It is not clear that we are going to show any of that at PDC. We are trying to be conservative in what we say at PDC for a change.
douglasp
12 Nov 09 at 11:50
Hi doug,
I kindof belong to both of these groups. You guys emphazise alot on “models are data”. And you’re right. But there is a huge difference between operational data as customers and orders and models. I know you have that vision about bluring the difference between runtime and compiletime, but lets face it - the nature of the data and the processes is totally different and therefore they need different treatment.
As for operational data, SQL is OK. We don’t need M. For operational data we care more about the actual storage than about the model-view on it. Additionally the datastructure doesn’t change too often, either. We think in transactions and use arbitrary locking mechanisms to handle concurrency issues. Concurrency issues on transactional data, BTW, is not a technical problem at all. The data and the backups are arround for decades. In this case the data is totally separate from the schema. We deploy schema changes, not data changes.
For models that define applications the story is totally different. This data is really just “code”. Whe need diff and merge, branch and tag. The deployment story is different, too. In this case the data often needs to be copied arround together with the schema. We also have need for migration-scenarios - but they are different. Backup is good, but the change history is much more important.
Also we are happy that M replaces XSD for schemas, and that in case we need it, this stuff is loaded into a database for faster queries. But really, we want a file story, too. Both for DSLs, grammars, schemas and unstructured M-values.
Maybe, some day, Visual Studio and all the C#-files are just models in a SQL Server Database. But that is still futures. Files won’t go away anytime soon, I guess. Isn’t a file-system a database?
I say “we” all time, because that is what I’ve heard from your potential customers all the way so far.
I have tons of more to say, but I guess I’ll write a blog entry, when I have more time.
- Lars
Lars Corneliussen
12 Nov 09 at 12:44
There was obviously a lot of confusion. Thank you for taking the time to clarify.
John
John
12 Nov 09 at 14:54
Was waiting for Lars to respond to your first “rename” post…looking forward for your blog post Lars, it will be an interesting discussion.
fallenprogrammr
12 Nov 09 at 15:09
I think some of us disagree with the statement that “Models are data”. (Yes it’s technically true, just like C# code can also be seen as data)
But you also have to consider the data’s use cases: To me, a model is a way to specify an application’s behavior at an higher level of abstraction than typical C# code, not a way to specify that a Customer has 1..n Telephone numbers.
A very convenient way of specifying behavior is with text files, which work great to specify complex logic, and play well with diff/merge/source control tools. It’s less convenient to have that behavior specified in a database. For example, I heard of third party products developed specifically for extracting an application from the Sharepoint DB into text files that can then be manipulated in a text editor and source controlled (and then imported back into Sharepoint). I don’t do Sharepoint development, but the existence of these tools leads me to believe that there’s something missing in the “application models as pure data” principle. (Would that be a gap filled by M?)
Also, I don’t see text files as an useful solution for specifying data: why would I want a Customer’s details in a plain text file? (maybe for loading test data for automated tests, but probably not for the real data)
On the other hand, having data in a relational structure allows very powerful and flexible reporting and data mining solutions. Now, would you do data mining or reporting on a model representing an application’s behavior? (not on the data that’s contained in instances of the model, but on the model itself?).
I can see some limited examples of “data mining” on application code, with tools like NDepend, but I would need more compelling exemples to convince me of the business value of models in the database. (Like another commenter said, it would help to have concrete examples of problems you are trying to solve)
Christos Karras
12 Nov 09 at 16:19
Totally agree with Lars.
Meta-data (model) requirements are different to operational data requirements. If the UML editor in Visual Studio was dependent on SQL Server then could you imagine the feedback?
Joe
12 Nov 09 at 18:38
Will the VS and .NET experience be tied in any way to the SQL Server license?
jedR
12 Nov 09 at 18:52
@Christos Karras
The text/database duality is one of the core problems that we are trying to solve. It was exactly what lead us to develop “M” in the first place.
In terms of scenarios for the repository, we think about a couple of core scenarios:
Application Structure/Development
1. storing conceputal, logical, physical models of the application definition
2. using these models to enable three key scenarios: application deployment/management, application understanding (architecture validation, application archeology, and impact analysis
Each of the above applies to the application architect, developer and operations.
We also think about key scenario for ISVs to build on, primary ISVs that already have a “repository”, like Active Directory.
There is a whole session at PDC about this.
I want to be clear, these are just scenarios for the repository.
“M” plays a role in this but it a general modeling lanuage.
douglasp
12 Nov 09 at 19:42
@jedr As I said before, there are lots of parts of SQL Server that we do not charge for. That said, we haven’t decided that part of the plan yet. We are focused on nailing the developer/business value of this right now.
douglasp
12 Nov 09 at 19:56
I was so happy to have Microsoft push for Oslo and DSL and at the same time embed UML in Visual Studio. I was really thinking that Microsoft will be the first one to offer a kind of pragmatic MDA for the mass. This could have revolutionized the way we do applications or services.
So the news about Oslo, even with your explanations that did not convince me, are very disappointing. Of course, Oslo market is not big enough to make it a full blown product. So, adding it to the DBMS/BI/MDM market is a smart move to apply the technology to bring competitive advantages.
I hope that it will enable a real Model driven approach for Master Data Management, and may be BI with SQL Server!
Anyway, Oslo will never be the same. That’s the law of product marketing and finance.
I’m so disapointed …
william el kaim
12 Nov 09 at 21:07
@colinjack
“M”
1. .NET Developers: Use “M” to create a model for a .NET application, which results in the database (schema, values, functions, .. ) and the EF types. Developers can then do whatever they want with the model; write a UI to work with it, write a service to do the same, etc.
2. .NET Developers: Use “M” to create textual DSLs over the model defined in #1.
3. Quadrant Developers: Use “M” to create and use models stored in databases and create custom viewers/behaviors
4. Quadrant Users: Use “M” to create queries and custom views over these models
“Quadrant”
1. Developers: Browse, edit, create models stored in databases (using “M”)
2. Developers: Provide custom viewers and behaviors over these models (using “M”)
3. Users: Use the custom viewers and behaviors over the models or use the in-built viewers/behaviors
“Repository”
1. Enterprise Architect/Dev/Operations: Storing conceptual, logical, and physical models of the application definition. Using these for architecture validation, application archeology, and impact analysis
2. ISVs: Many ISVs (Active Directory, SharePoint, Dynamics, etc.) already have a “repository”. Our repository is a turn-key solution for these customers. This turn-key solution is designed to save these ISVs effort and to allow us (and them)to have a great deal of sharing between these implementations
We are going to show all of this at PDC and you can use the CTP to all of these scenarios as well.
douglasp
13 Nov 09 at 04:11
Doug - Could you answer this? Will I still be able to use Intellipad to define “M” artifacts, add a reference to the “M” parser in my .NET projects in order to create DynamicParsers and instances of MGraphs, redistribute those projects, and build additional tooling around “M”, *WITHOUT* having to license, reference, or even think about anything to do with SQL Server or databases in general with this new found direction?
If the answer is yes, although I don’t agree with this change in direction, I can live with it and still see a use for some aspects of the formally Oslo initiative.
If the answer is no, then sadly for many of us in the community, myself included, the #fail moniker aptly applies.
Jordan Terrell
13 Nov 09 at 16:55
[...] On DSLs and a few other things and On "M" - Doug Purdy provides clarifications and pushback on some of the feedback from the From “Oslo” to SQL Server Modeling, which seems to be mostly negative [...]
New and Notable 390 : Sam Gentile's Blog (if (DeveloperTask == Communication && OS == Windows)
16 Nov 09 at 20:03
[...] More information:http://www.douglaspurdy.com/2009/11/10/from-oslo-to-sql-server-modeling/http://www.douglaspurdy.com/2009/11/12/on-dsls-and-a-few-other-things/http://social.msdn.microsoft.com/Forums/en-US/oslo/threads/ [...]
SQL Server Modeling - the new name for Oslo - Johan Åhlén
17 Nov 09 at 17:56
I agree with Jordan that it is now a question of what the name change means in practise. If you can use the DSL / MGrammar functionality without purchasing SQL Server licenses I think it is okay. Otherwise I think you will loose a lot of community support.
JAhlen
17 Nov 09 at 18:09