domain-specific language

40 results back to index


pages: 448 words: 71,301

Programming Scala by Unknown

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

domain-specific language, en.wikipedia.org, fault tolerance, general-purpose programming language, loose coupling, type inference, web application

Finally, note that when you use the scala command, either interactively or with scripts, it may appear that you can define “global” variables and methods outside of types. This is actually an illusion; the interpreter wraps all definitions in an anonymous type before generating JVM or .NET CLR byte code. Domain-Specific Languages Domain-Specific Languages, or DSLs, provide a convenient syntactical means for expressing goals in a given problem domain. For example, SQL provides just enough of a programming language to handle the problems of working with databases, making it a Domain-Specific Language. While some DSLs like SQL are self-contained, it’s become popular to implement DSLs as subsets of full-fledged programming languages. This allows programmers to leverage the entirety of the host language for edge cases that the DSL does not cover, and saves the work of writing lexers, parsers, and the other building blocks of a language.

Writing XML with Scala is straightforward: construct the document you need with inline XML, use interpolation where dynamic content is to be substituted, and make use of the handy convenience methods to write your completed documents to disk or to other output streams. Recap and What’s Next XML has become ubiquitous in software applications, yet few languages make working with XML a simple task. We learned how Scala accelerates XML development by making it easy to read and write XML. In the next chapter, we’ll learn how Scala provides rich support for creating your own Domain-Specific Languages (DSLs). 216 | Chapter 10: Herding XML in Scala Download at WoweBook.Com CHAPTER 11 Domain-Specific Languages in Scala A Domain-Specific Language is a programming language that mimics the terms, idioms, and expressions used among experts in the targeted domain. Code written in a DSL reads like structured prose for the domain. Ideally, a domain expert with little experience in programming can read, understand, and validate this code. Sometimes, a domain expert might be able to write DSL code, even if he isn’t a professional programmer.

[DesignByContract] Building bug-free O-O software: An introduction to Design by Contract™, http://archive.eiffel.com/doc/manuals/technology/contract/. 387 Download at WoweBook.Com [Deursen] Arie van Deursen, Paul Klint, and Joost Visser, Domain-Specific Languages: An Annotated Bibliography, http://homepages.cwi.nl/~arie/papers/dslbib/. [EBNF] Extended Backus-Naur Form, http://en.wikipedia.org/wiki/Extended_Backus– Naur_Form. [Eiffel] Eiffel Software, http://eiffel.com. [Ford] Bryan Ford, The Packrat Parsing and Parsing Expression Grammars Page, http: //pdos.csail.mit.edu/~baford/packrat/. [Ford2009] Neal Ford, Advanced DSLs in Ruby, http://github.com/nealford/presenta tions/tree/master. [Fowler2009] Martin Fowler, Domain Specific Languages (forthcoming), http://martin fowler.com/dslwip/. [FunctionalJava] Functional Java, http://functionaljava.org. [Ghosh2008a] Debasish Ghosh, External DSLs made easy with Scala Parser Combinators, http://debasishg.blogspot.com/2008/04/external-dsls-made-easy-with-scala.html.


pages: 292 words: 62,575

97 Things Every Programmer Should Know by Kevlin Henney

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

A Pattern Language, active measures, business intelligence, commoditize, continuous integration, crowdsourcing, database schema, deliberate practice, domain-specific language, don't repeat yourself, Donald Knuth, fixed income, general-purpose programming language, Grace Hopper, index card, inventory management, job satisfaction, loose coupling, Silicon Valley, sorting algorithm, The Wisdom of Crowds

(You Are Not the User) Prevent Errors, Prevent Errors, Prevent Errors, Prevent Errors collaboration, Two Heads Are Often Better Than One, Two Heads Are Often Better Than One, Two Heads Are Often Better Than One, When Programmers and Testers Collaborate, When Programmers and Testers Collaborate, When Programmers and Testers Collaborate testers and programmers, When Programmers and Testers Collaborate, When Programmers and Testers Collaborate, When Programmers and Testers Collaborate Colvin, Know Your Limits, Know Your Limits, Know Your Limits Know Your Limits, Know Your Limits, Know Your Limits, Know Your Limits command-line build tools, Know How to Use Command-Line Tools, Know How to Use Command-Line Tools, Know How to Use Command-Line Tools comments, A Comment on Comments, Comment Only What the Code Cannot Say, Only the Code Tells the Truth, Write Code As If You Had to Support It for the Rest of Your Life committing code, Test While You Sleep (and over Weekends) communicating with peers, Learn Foreign Languages, Learn Foreign Languages, Learn Foreign Languages compiler bugs, Check Your Code First Before Looking to Blame Others components, Choose Your Tools with Care concurrency, Message Passing Leads to Better Scalability in Parallel Systems, Message Passing Leads to Better Scalability in Parallel Systems, Message Passing Leads to Better Scalability in Parallel Systems continuous learning, Continuous Learning, Continuous Learning, Continuous Learning contradictions in code, Two Wrongs Can Make a Right (and Are Difficult to Fix), Two Wrongs Can Make a Right (and Are Difficult to Fix), Two Wrongs Can Make a Right (and Are Difficult to Fix) correctness of code, Coding with Reason, Coding with Reason, Coding with Reason CVS, Put Everything Under Version Control D Dahan, Beware the Share, Beware the Share, Beware the Share Beware the Share, Beware the Share, Beware the Share, Beware the Share data structures, Interprocess Communication Affects Application Response Time using caches, Interprocess Communication Affects Application Response Time deliberate practice, Do Lots of Deliberate Practice, Do Lots of Deliberate Practice, Do Lots of Deliberate Practice deployment process, Deploy Early and Often, Deploy Early and Often, Deploy Early and Often design, Code Is Design, Code Is Design, Code Is Design dirty code, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs Doar, How to Use a Bug Tracker, How to Use a Bug Tracker, How to Use a Bug Tracker How to Use a Bug Tracker, How to Use a Bug Tracker, How to Use a Bug Tracker, How to Use a Bug Tracker domain-specific languages (DSLs), Domain-Specific Languages, Domain-Specific Languages, Domain-Specific Languages domain-specific typing, Prefer Domain-Specific Types to Primitive Types, Prefer Domain-Specific Types to Primitive Types, Prefer Domain-Specific Types to Primitive Types DRY (Don't Repeat Yourself), Don't Repeat Yourself, WET Dilutes Performance Bottlenecks duplication, Don't Repeat Yourself, Own (and Refactor) the Build E encapsulation, Encapsulate Behavior, Not Just State, Encapsulate Behavior, Not Just State, Encapsulate Behavior, Not Just State Erlang, Message Passing Leads to Better Scalability in Parallel Systems errno, Don't Ignore That Error!

And Mary Poppendieck writes: Deliberate practice does not mean doing what you are good at; it means challenging yourself, doing what you are not good at. So it's not necessarily fun. Deliberate practice is about learning—learning that changes you, learning that changes your behavior. Good luck. * * * [4] http://norvig.com/21-days.html Chapter 23. Domain-Specific Languages Michael Hunger WHENEVER YOU LISTEN TO A DISCUSSION BY EXPERTS in any domain, be it chess players, kindergarten teachers, or insurance agents, you'll notice that their vocabulary is quite different from everyday language. That's part of what domain-specific languages (DSLs) are about: a specific domain has a specialized vocabulary to describe the things that are particular to that domain. In the world of software, DSLs are about executable expressions in a language specific to a domain, employing a limited vocabulary and grammar that is readable, understandable, and—hopefully—writable by domain experts.

A acceptance test-driven development (ATDD), When Programmers and Testers Collaborate Ada, Know Well More Than Two Programming Languages, Prefer Domain-Specific Types to Primitive Types Adams, Make the Invisible More Visible algorithms, Interprocess Communication Affects Application Response Time, Know Your Limits limits, Know Your Limits performance, Interprocess Communication Affects Application Response Time Allison, Floating-Point Numbers Aren't Real, Floating-Point Numbers Aren't Real, Floating-Point Numbers Aren't Real Floating-Point Numbers Aren't Real, Floating-Point Numbers Aren't Real, Floating-Point Numbers Aren't Real, Floating-Point Numbers Aren't Real analysis tools, Take Advantage of Code Analysis Tools APIs, Convenience Is Not an -ility, Convenience Is Not an -ility, Convenience Is Not an -ility, Domain-Specific Languages, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design API design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design, The Golden Rule of API Design internal DSLs, Domain-Specific Languages Asproni, Choose Your Tools with Care, Choose Your Tools with Care, Choose Your Tools with Care, Learn to Estimate, Learn to Estimate, Learn to Estimate Choose Your Tools with Care, Choose Your Tools with Care, Choose Your Tools with Care, Choose Your Tools with Care Learn to Estimate, Learn to Estimate, Learn to Estimate, Learn to Estimate Attapattu, Before You Refactor, Before You Refactor, Before You Refactor, Test While You Sleep (and over Weekends), Test While You Sleep (and over Weekends), Test While You Sleep (and over Weekends) Before You Refactor, Before You Refactor, Before You Refactor, Before You Refactor Test While You Sleep (and over Weekends), Test While You Sleep (and over Weekends), Test While You Sleep (and over Weekends), Test While You Sleep (and over Weekends) automation, Automate Your Coding Standard, Automate Your Coding Standard, Automate Your Coding Standard, Step Back and Automate, Automate, Automate, Step Back and Automate, Automate, Automate, Step Back and Automate, Automate, Automate automating coding standard, Automate Your Coding Standard, Automate Your Coding Standard, Automate Your Coding Standard B Baden-Powell, The Boy Scout Rule Baker, Install Me, Install Me, Install Me Install Me, Install Me, Install Me, Install Me beautiful code, Beauty Is in Simplicity, A Message to the Future Begbie, Don't Be Cute with Your Test Data, Don't Be Cute with Your Test Data, Don't Be Cute with Your Test Data Don't Be Cute with Your Test Data, Don't Be Cute with Your Test Data, Don't Be Cute with Your Test Data, Don't Be Cute with Your Test Data Berczuk, Deploy Early and Often, Deploy Early and Often, Deploy Early and Often, Deploy Early and Often Deploy Early and Often, Deploy Early and Often, Deploy Early and Often, Deploy Early and Often, Deploy Early and Often Berg, Read Code, Read Code, Read Code Read Code, Read Code, Read Code, Read Code binary, One Binary blaming others, How to Use a Bug Tracker bottlenecks, Code Reviews, Interprocess Communication Affects Application Response Time, WET Dilutes Performance Bottlenecks, WET Dilutes Performance Bottlenecks, WET Dilutes Performance Bottlenecks performance, Interprocess Communication Affects Application Response Time, WET Dilutes Performance Bottlenecks, WET Dilutes Performance Bottlenecks, WET Dilutes Performance Bottlenecks process, Code Reviews Boy Scout rule, The Boy Scout Rule, The Boy Scout Rule, The Boy Scout Rule Braithwaite, Don't Be Afraid to Break Things, The Linker Is Not a Magical Program, The Linker Is Not a Magical Program, The Linker Is Not a Magical Program, Read the Humanities, Read the Humanities, Read the Humanities, Write Small Functions Using Examples breaking code, Don't Be Afraid to Break Things Read the Humanities, Read the Humanities, Read the Humanities, Read the Humanities The Linker Is Not a Magical Program, The Linker Is Not a Magical Program, The Linker Is Not a Magical Program, The Linker Is Not a Magical Program Write Small Functions Using, Write Small Functions Using Examples Examples, Write Small Functions Using Examples Brodwall, Keep the Build Clean, Keep the Build Clean, Keep the Build Clean Keep the Build Clean, Keep the Build Clean, Keep the Build Clean, Keep the Build Clean Brush, Code Is Design, Code Is Design, Code Is Design, The Guru Myth, The Guru Myth, The Guru Myth Code Is Design, Code Is Design, Code Is Design, Code Is Design The Guru Myth, The Guru Myth, The Guru Myth, The Guru Myth bug tracker, How to Use a Bug Tracker, How to Use a Bug Tracker, How to Use a Bug Tracker build scripts, Own (and Refactor) the Build, Own (and Refactor) the Build, Own (and Refactor) the Build builds, Keep the Build Clean keeping clean, Keep the Build Clean C C#, Distinguish Business Exceptions from Technical, Don't Just Learn the Language, Understand Its Culture, The Golden Rule of API Design C++, Don't Nail Your Program into the Upright Position, Know Well More Than Two Programming Languages caller breaking method contract, Distinguish Business Exceptions from Technical COBOL, Code Layout Matters, Don't Rely on "Magic Happens Here" code, The Boy Scout Rule, Check Your Code First Before Looking to Blame Others, Check Your Code First Before Looking to Blame Others, Check Your Code First Before Looking to Blame Others, Code Is Design, Code Layout Matters, Code Reviews, Coding with Reason, Coding with Reason, Coding with Reason, Comment Only What the Code Cannot Say, Domain-Specific Languages, Don't Repeat Yourself, Don't Repeat Yourself, Only the Code Tells the Truth, Only the Code Tells the Truth, Only the Code Tells the Truth, Only the Code Tells the Truth, Read Code, Read Code, Read Code, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs, Take Advantage of Code Analysis Tools, Take Advantage of Code Analysis Tools, Take Advantage of Code Analysis Tools, Test While You Sleep (and over Weekends), Testing Is the Engineering Rigor of Software Development, Write Code As If You Had to Support It for the Rest of Your Life, Write Code As If You Had to Support It for the Rest of Your Life, Write Code As If You Had to Support It for the Rest of Your Life, You Gotta Care About the Code, You Gotta Care About the Code, You Gotta Care About the Code analysis tools, Take Advantage of Code Analysis Tools, Take Advantage of Code Analysis Tools, Take Advantage of Code Analysis Tools as design, Code Is Design, Testing Is the Engineering Rigor of Software Development caring about, You Gotta Care About the Code, You Gotta Care About the Code, You Gotta Care About the Code checking code before blaming others, Check Your Code First Before Looking to Blame Others, Check Your Code First Before Looking to Blame Others, Check Your Code First Before Looking to Blame Others clear information, Only the Code Tells the Truth, Only the Code Tells the Truth, Only the Code Tells the Truth comments, Comment Only What the Code Cannot Say, Only the Code Tells the Truth dirty, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs, The Road to Performance Is Littered with Dirty Code Bombs domain concepts, Domain-Specific Languages duplication, Don't Repeat Yourself libraries of shared code, The Boy Scout Rule reading, Code Layout Matters, Read Code, Read Code, Read Code reasoning semiformally about correctness, Coding with Reason, Coding with Reason, Coding with Reason repetition, Don't Repeat Yourself process calls, Don't Repeat Yourself reviews, Code Reviews running tests before committing code, Test While You Sleep (and over Weekends) writing and supporting, Write Code As If You Had to Support It for the Rest of Your Life, Write Code As If You Had to Support It for the Rest of Your Life, Write Code As If You Had to Support It for the Rest of Your Life codebase, Keep the Build Clean coding practices, Coding with Reason, Coding with Reason, Coding with Reason Colborne, Ask, "What Would the User Do?"


pages: 496 words: 174,084

Masterminds of Programming: Conversations With the Creators of Major Programming Languages by Federico Biancuzzi, Shane Warden

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Benevolent Dictator For Life (BDFL), business intelligence, business process, cellular automata, cloud computing, commoditize, complexity theory, conceptual framework, continuous integration, data acquisition, domain-specific language, Douglas Hofstadter, Fellow of the Royal Society, finite state, Firefox, follow your passion, Frank Gehry, general-purpose programming language, Guido van Rossum, HyperCard, information retrieval, iterative process, John von Neumann, Larry Wall, linear programming, loose coupling, Mars Rover, millennium bug, NP-complete, Paul Graham, performance metric, Perl 6, QWERTY keyboard, RAND corporation, randomized controlled trial, Renaissance Technologies, Ruby on Rails, Sapir-Whorf hypothesis, Silicon Valley, slashdot, software as a service, software patent, sorting algorithm, Steve Jobs, traveling salesman, Turing complete, type inference, Valgrind, Von Neumann architecture, web application

How would you “restructure UML as a set of domain-specific languages”? Ivar: We have a basic universally applicable core in UML. I would identify aspects of that core, and describe UML by adding aspect after aspect. These aspects of UML are what we call practices when it comes to processes, and these practice-like things of UML would be domain-specific languages. A domain-specific language would, as the name suggests, support a particular domain, such as a vertical industry sector (enterprise systems, telecom systems, healthcare systems, etc.) or a discipline (requirements, design, real-time, test, etc.). A rather small aspect of UML would constitute one domain-specific language. In that way you would compose UML from different domain-specific languages. These domain-specific languages need to have a common core and a common semantic, otherwise we’ll have a very difficult problem in translating between things in different domains.

Then, of course, there comes a time when you just can’t stuff more in there—there’s so much tension between the new things you add and the old way of doing it in the language that you just can’t move it anymore. Creating a new language is really more of an exception to the rule than it is the rule. Would you create a general-purpose language or a domain-specific language? Anders: I think the real answer there is “neither.” How I would address that problem is I would create a general-purpose programming language that is great at creating domain-specific languages. Again, the devil that we face with all of these domain-specific languages is that they may get the domain right but then they get the general-purposeness wrong. There are certain general-purpose features that literally every domain-specific language ends up needing. Unless the domain-specific language is purely just a data definition language where you’re just stating data, and at that point in my opinion you might as well use XML then. If you’re really a programming language where there’s logic or predicates or rules or whatever, then you have to have expressions and expressions have operators and maybe you have to have standard functions and your customers are going to want to do things that you never even thought of.

That’s kind of the world we’re in right now. Even the high priests at times get surprised by their own code today. We have a big challenge there. Also there’s a lot of talk about domain-specific languages and metaprogramming in the community these days. In my opinion, there’s more talk than reality there. I don’t think we know what the answers are. You see things like aspect-oriented programming and intentional programming, but we have yet to really nail it. Depending on who you ask, people go either, “Oh, there are no domain-specific languages,” or “Oh, domain-specific languages are everywhere.” We can’t even agree on what a domain-specific language is, in a sense—but there’s clearly a there there when it comes to devising more declarative ways of expressing yourself. In some ways, we’ve run out the line all the way on imperative styles of programming.


pages: 224 words: 48,804

The Productive Programmer by Neal Ford

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

anti-pattern, business process, c2.com, continuous integration, database schema, domain-specific language, don't repeat yourself, Firefox, general-purpose programming language, knowledge worker, Larry Wall, Ruby on Rails, side project, type inference, web application, William of Occam

This diagram suggests that we’ll use a language (perhaps a statically typed language) for the stability layer, a more productive language (probably dynamic in nature, like JRuby, Groovy, or Jython) for day-to-day coding, and domain-specific languages (as discussed in “Fluent Interfaces” in Chapter 11) to align our code more closely to the needs of business analysts and end users. I think Ola has nailed the way that the disparate ideas of polyglot programming, domain-specific languages, and dynamic languages all fit together. All doctors at one time were general practitioners, but as their field advanced, specialization became inevitable. Software complexity is quickly driving us toward specialization, both because of the types of applications we must write and the underlying platforms. To address this brave new world, we should embrace polyglot programming to provide more specialized tools for the platforms, and domain-specific languages to address increasingly difficult problem domains.

Instead of keeping developers out of trouble, the accidental complexity of modern languages has forced developers to wade through lots of complicated workarounds. This trend has a negative impact on productivity when building complex software. What we really want is the productivity of 4GLs with the generality and flexibility of powerful general-purpose languages. Enter frameworks built with domain-specific languages (DSLs), the current exemplar being Ruby on Rails. When writing a Rails application, you don’t write that much “pure” Ruby code (and most of that is in models, for business rules). Mostly, you are writing code in the DSL part of Rails. That means that you get major bang for the buck: validates_presence_of :name, :sales_description, :logo_image_url validates_numericality_of :account_balance validates_uniqueness_of :name validates_format_of :logo_image_url, Occam’s Razor 139 :with => %r{\.

Note that this isn’t necessarily a dynamic language or even a Ruby tirade: a strong potential exists for statically typed type-inference languages that have a suitable syntax to also take advantage of this style of programming. For an example of this, check out Jaskell* and, in particular, the build DSL written on top of it called Neptune.† Neptune performs the same basic tasks as Ant, but it is written as a domain-specific language atop Jaskell. It shows how readable and concise you can make code in Jaskell, using a familiar problem domain. NOTE Dietzler’s Law: even general-purpose programming languages suffer from the “80-10-10” rule. The Law of Demeter The Law of Demeter was developed at Northwestern University in the late ’80s. It is best summarized by the phrase, “Only talk to your closest friends.” The idea is that any given object shouldn’t know anything about the internal details of the objects with which it interacts.


pages: 462 words: 172,671

Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

continuous integration, database schema, domain-specific language, don't repeat yourself, Donald Knuth, en.wikipedia.org, Eratosthenes, finite state, Ignaz Semmelweis: hand washing, iterative process, place-making, Rubik’s Cube, web application, WebSocket

At some level those details cannot be ignored or abstracted; they have to be specified. And specifying requirements in such detail that a machine can execute them is programming. Such a specification is code. I expect that the level of abstraction of our languages will continue to increase. I also expect that the number of domain-specific languages will continue to grow. This will be a good thing. But it will not eliminate code. Indeed, all the specifications written in these higher level and domain-specific language will be code! It will still need to be rigorous, accurate, and so formal and detailed that a machine can understand and execute it. The folks who think that code will one day disappear are like mathematicians who hope one day to discover a mathematics that does not have to be formal. They are hoping that one day we will discover a way to create machines that can do what we want rather than what we say.

I don’t think anyone could. Conclusion Every system is built from a domain-specific language designed by the programmers to describe that system. Functions are the verbs of that language, and classes are the nouns. This is not some throwback to the hideous old notion that the nouns and verbs in a requirements document are the first guess of the classes and functions of a system. Rather, this is a much older truth. The art of programming is, and has always been, the art of language design. Master programmers think of systems as stories to be told rather than programs to be written. They use the facilities of their chosen programming language to construct a much richer and more expressive language that can be used to tell that story. Part of that domain-specific language is the hierarchy of functions that describe all the actions that take place within that system.

Standards make it easier to reuse ideas and components, recruit people with relevant experience, encapsulate good ideas, and wire components together. However, the process of creating standards can sometimes take too long for industry to wait, and some standards lose touch with the real needs of the adopters they are intended to serve. Systems Need Domain-Specific Languages Building construction, like most domains, has developed a rich language with a vocabulary, idioms, and patterns21 that convey essential information clearly and concisely. In software, there has been renewed interest recently in creating Domain-Specific Languages (DSLs),22 which are separate, small scripting languages or APIs in standard languages that permit code to be written so that it reads like a structured form of prose that a domain expert might write. 21. The work of [Alexander] has been particularly influential on the software community. 22.


pages: 754 words: 48,930

Programming in Scala by Martin Odersky, Lex Spoon, Bill Venners

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

domain-specific language, Guido van Rossum, Larry Wall, Silicon Valley, sorting algorithm, type inference, web application

This might be the best solution, except for a couple of inconveniences. You need to learn new tools, including their—sometimes obscure—error messages. You also need to figure out how to connect the output of these tools to your program. This might limit the choice of your programming language, and complicate your tool chain. This chapter presents a third alternative. Instead of using the standalone domain specific language of a parser generator, you will use an internal domain specific language, or internal DSL for short. The internal DSL will consist of a library of parser combinators—functions and operators defined in Scala that will serve as building blocks for parsers. These building blocks Cover · Overview · Contents · Discuss · Suggest · Glossary · Index Section 31.1 Chapter 31 · Combinator Parsing will map one to one to the constructions of a context-free grammar, to make them easy to understand.

.), 297n Dijkstra, Edsger, 54 Dirty Harry, 620 divide and conquer principle, 333 do-while loops, 152 domain objects, 555 Domain-Driven Design (Evans), 555n domain-specific languages, 707 a little language for digital circuits, 393 and implicit conversions, 473 defining an external DSL with parser combinators, 642–652 embedded in Scala, xxvii Cover · Overview · Contents · Discuss · Suggest · Glossary · Index Index in Smalltalk and Lisp, 56 parser combinators as an internal DSL, 642 rich wrappers and internal versus external DSLs, 475 Double class, 107 downloading Scala, 58 source code for book examples, xxxvii drop method on class List, 78, 337 dropRight method on class List, 78 dropWhile method on class List, 346 DSLs, see domain-specific languages dynamic binding, 222, 248 typing, xxvii, 53, 114 E Eastwood, Clint, 620 Eckel, Bruce, 226n Eclipse IDE, 58, 671 EditDone class, 681 Effective C++ (Meyers), 226n Effective Java, Second Edition (Bloch), 569n, 582n, 590n efficiency and hash consing, 242 and trait ScalaObject, 241 of abstract classes versus traits, 263n, 294n of arrays, 75, 362 of case classes versus extractors, 526 of class ListBuffer, 363, 494 of fold left and right, 349 of functional on the outside, imperative on the inside approach, 497 of immutable objects, 131 737 of lists, 361 of LL(1) versus backtracking parsers, 669 of looping versus recursion, 189 of maps, 373 of merge versus insertion sort, 341 of sets, 373 of tail versus non-tail recursion, 495 of thread creation and switching, 613 of value types, 59, 75, 238, 595 Eiffel language, 56 Elem class, 541, 546 Elem type in trait Parsers, 657 elements method on class List, 341 on trait Iterable, 359 elements, XML, 540–541 else clauses, 150 embedded DSLs, see domain-specific languages empty method on Map objects, 369, 373 on Set objects, 369, 370 empty-paren methods, 212 enclosing instances, see inner classes enqueue method on class Queue, 365 ensuring method on object Predef, 280 entities, 556 enum keyword (Java), see Enumeration class Enumeration class, 452 eq method on class AnyRef, 124, 243, 568 equality, see also equals method !

Scala takes powerful features from object-oriented and functional languages, and combines them with a few novel ideas in a beautifully coherent whole. The syntax is so lightweight, and its primitives so expressive, that APIs can be used with virtually no syntactic overhead at all. Examples can be found in standard libraries such as parser combinators and actors. In this sense Scala supports embedded domain-specific languages. Cover · Overview · Contents · Discuss · Suggest · Glossary · Index Foreword xxviii Will Scala be the next great language? Only time will tell. Martin Odersky’s team certainly has the taste and skill for the job. One thing is sure: Scala sets a new standard against which future languages will be measured. Neal Gafter San Jose, California September 3, 2008 Cover · Overview · Contents · Discuss · Suggest · Glossary · Index Acknowledgments Many people have contributed to this book and to the material it covers.


pages: 706 words: 120,784

The Joy of Clojure by Michael Fogus, Chris Houser

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

cloud computing, domain-specific language, Donald Knuth, Douglas Hofstadter, en.wikipedia.org, finite state, Gödel, Escher, Bach, haute couture, Larry Wall, Paul Graham, rolodex, traveling salesman

It’s an example that we’ll get working on in a moment: (defn query [max] (SELECT [a b c] (FROM X (LEFT-JOIN Y :ON (= X.a Y.b))) (WHERE (AND (< a 5) (< b ~max))))) We hope some of those words look familiar to you, because this isn’t a book on SQL. Regardless, our point here is that Clojure doesn’t have SQL support built in. The words SELECT, FROM, and so forth aren’t built-in forms. They’re also not regular functions, because if SELECT were, then the use of a, b, and c would be an error, because they haven’t been defined yet. So what does it take to define a domain-specific language (DSL) like this in Clojure? Well, it’s not production-ready code and doesn’t tie into any real database servers; but with just one macro and the three functions shown in listing 1.1, the preceding query returns these handy values: (query 5) ;=> ["SELECT a, b, c FROM X LEFT JOIN Y ON (X.a = Y.b) WHERE ((a < 5) AND (b < ?))" [5]] Note that some words such as FROM and ON are taken directly from the input expression, whereas others such as ~max and AND are treated specially.

[5]] Note that some words such as FROM and ON are taken directly from the input expression, whereas others such as ~max and AND are treated specially. The max that was given the value 5 when the query was called is extracted from the literal SQL string and provided in a separate vector, perfect for using in a prepared query in a way that will guard against SQL-injection attacks. The AND form was converted from the prefix notation of Clojure to the infix notation required by SQL. Listing 1.1. A domain-specific language for embedding SQL queries in Clojure But the point here isn’t that this is a particularly good SQL DSL—more complete ones are available.[4] Our point is that once you have the skill to easily create a DSL like this, you’ll recognize opportunities to define your own that solve much narrower, application-specific problems than SQL does. Whether it’s a query language for an unusual non-SQL datastore, a way to express functions in some obscure math discipline, or some other application we as authors can’t imagine, having the flexibility to extend the base language like this, without losing access to any of the language’s own features, is a game-changer. 4 One of note is ClojureQL at http://gitorious.org/clojureql.

Where macros differ from techniques familiar to proponents of Java’s object-oriented style—including hierarchies, frameworks, inversion of control, and the like—is that they’re treated no differently by the language itself. Clojure macros work to mold the language into the problem space rather than forcing you to mold the problem space into the constructs of the language. There’s a specific term for this, domain-specific language, but in Lisp the distinction between DSL and API is thin to the point of transparency. Envision a scenario where you want to be able to define Vars that call a function whenever their root bindings change. You could do this using the add-watch function that allows for the attachment of a watcher to a reference type that’s called whenever a change occurs within. The add-watch function itself takes three arguments: a reference, a watch function key, and a watch function called whenever a change occurs.


pages: 1,065 words: 229,099

Real World Haskell by Bryan O'Sullivan, John Goerzen, Donald Stewart, Donald Bruce Stewart

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

bash_history, database schema, Debian, digital map, distributed revision control, domain-specific language, en.wikipedia.org, Firefox, general-purpose programming language, Guido van Rossum, job automation, Larry Wall, p-value, Plutocrats, plutocrats, revision control, sorting algorithm, transfer pricing, type inference, web application, Y Combinator

, Tuning for Performance parallel programs, Defining Concurrency and Parallelism, Parallel Programming in Haskell–Parallel Strategies and MapReduce, Parallel Strategies and MapReduce MapReduce and, Parallel Strategies and MapReduce parameterized types, Polymorphism in Haskell, Parameterized Types parametric polymorphism, Polymorphism in Haskell parentheses (( )), Simple Arithmetic, Operator Precedence and Associativity, Useful Composite Data Types: Lists and Tuples, Folding from the Right arithmetic expressions, writing, Simple Arithmetic foldl and foldr functions, Folding from the Right operator precedence and, Operator Precedence and Associativity tuples, writing, Useful Composite Data Types: Lists and Tuples Parse type, Writing a Functor Instance for Parse, Motivation: Boilerplate Avoidance, Replacing the Parse Type with a Monad Stack monad stack, replacing, Replacing the Parse Type with a Monad Stack Parse wrapper, The Identity Parser parseByte function, Obtaining and Modifying the Parse State Parsec, Using Parsec–Parsing Headers, First Steps with Parsec: Simple CSV Parsing–The sepBy and endBy Combinators, Parsec and MonadPlus CSV files and, First Steps with Parsec: Simple CSV Parsing–The sepBy and endBy Combinators MonadPlus and, Parsec and MonadPlus Parsec CSV files and, The sepBy and endBy Combinators parseChar function, Using Functors for Parsing parseTimes function, Parsing a Color Image partial application of functions, Partial Function Application and Currying partial functions, Partial and Total Functions partial type Monad instances, Understanding Monad Transformers by Building One pattern matches, The Wild Card Pattern, Differences Between Data and Newtype Declarations newtype declarations and, Differences Between Data and Newtype Declarations wild card, The Wild Card Pattern pattern matching, Pattern Matching–Record Syntax, Variable Naming in Patterns, As-patterns, Filename Matching, Record Syntax, Updates, and Pattern Matching, Pattern Matching with Substrings as-patterns, As-patterns filenames, Filename Matching record syntax and, Record Syntax, Updates, and Pattern Matching substrings, binding C with Haskell, Pattern Matching with Substrings variable naming, Variable Naming in Patterns PatternGuards language extension, Pattern Guards PCRE (Perl-compatible regular expression), Interfacing with C: The FFI, Simple Tasks: Using the C Preprocessor, Adding Type Safety to PCRE C preprocessor and, Simple Tasks: Using the C Preprocessor types, adding safety to, Adding Type Safety to PCRE pcre.h header file, Passing String Data Between Haskell and C pcre_compile function, Allocating Local C Data: The Storable Class pcre_exec function, Matching on Strings pcre_fullinfo function, Extracting Information About the Pattern peek function, Passing String Data Between Haskell and C Perl, Other Things You Should Know, Interfacing with C: The FFI PCRE library and, Interfacing with C: The FFI regular expressions and, Other Things You Should Know permissive types, Strong Types PGM files, Parsing a Raw PGM File–Getting Rid of Boilerplate Code pi value, Numeric Types pipe (|), separating value constructors, Algebraic Data Types pipes (redirection), Using Pipes for Redirection–Final Words on Pipes pivots, Sequential Sorting pixels, Grayscale to Binary and Type Safety plain (P3) netpbm format, Grayscale Files plusPtr function, Passing String Data Between Haskell and C podcatcher, building, Extended Example: Web Client Programming point (.) character, Pretty Printing a String point-free expressions, Pretty Printing a String point-free style, Pretty Printing a String pointers, Passing String Data Between Haskell and C, Typed Pointers typed, Typed Pointers poke function, Passing String Data Between Haskell and C polymorphic list types, Useful Composite Data Types: Lists and Tuples polymorphic testing, Polymorphic Testing polymorphic types, Polymorphism in Haskell, Parameterized Types polymorphism, Polymorphism in Haskell–The Type of a Function of More Than One Argument positional types, The structure POSIX regular expressions, Other Things You Should Know POST method (HTTP), Parsing a HTTP Request PostgreSQL, Overview of HDBC predicates, Conditional Evaluation, Predicates: From Poverty to Riches, While Remaining Pure–Sizing a File Safely, A Domain-Specific Language for Predicates–Controlling Traversal domain-specific language for, A Domain-Specific Language for Predicates–Controlling Traversal I/O, Predicates: From Poverty to Riches, While Remaining Pure–Sizing a File Safely prefix form, Simple Arithmetic prefix notation, Infix Functions Prelude module, Getting Started with ghci, the Interpreter, Important Built-in Typeclasses prepare function, Prepared Statements pretty printers, A More General Look at Rendering, Testing Case Study: Specifying a Pretty Printer–Measuring Test Coverage with HPC testing, Testing Case Study: Specifying a Pretty Printer–Measuring Test Coverage with HPC primitive recursive functions, Folding from the Right print function, Working with Files and Handles printf (C) option, Profiling Haskell Programs PRNG (pseudorandom number generator), Running Our Code and Measuring Performance Product IDs of barcodes, A Little Bit About Barcodes product types, Generating Test Data -prof flag (GHC), Time Profiling profiling, Profiling Haskell Programs program termination, Program Termination programmable semicolon, Monads as a Programmable Semicolon programming environment, Your Haskell Environment projection functions, Working with Row Data prompt directive (ghci), Getting Started with ghci, the Interpreter property-based testing, QuickCheck: Type-Based Testing prop_one_present function, Polymorphic Testing pseudorandom number generator (PRNG), Running Our Code and Measuring Performance pseudorandom value generation, Using the State Monad: Generating Random Values Ptr type, Passing String Data Between Haskell and C pure code, Interference with Pure Code pure error handling, Error Handling pure function, Function Types and Purity, Pure Versus I/O, Parsing a Raw PGM File PGM files and, Parsing a Raw PGM File vs.

The former ensures that we don’t accumulate garbage file handles, while the latter gets rid of exceptions. Exercise Is the order in which we call bracket and handle important? Why? A Domain-Specific Language for Predicates Let’s take a stab at writing a predicate that will check for a C++ source file that is over 128 KB in size: -- file: ch09/BetterPredicate.hs myTest path _ (Just size) _ = takeExtension path == ".cpp" && size > 131072 myTest _ _ _ _ = False This isn’t especially pleasing. The predicate takes four arguments, always ignores two of them, and requires two equations to define. Surely we can do better. Let’s create some code that will help us write more concise predicates. Sometimes, this kind of library is referred to as an embedded domain-specific language: we use our programming language’s native facilities (hence embedded) to write code that lets us solve some narrow problem (hence domain-specific) particularly elegantly.

combinator functions, Gluing Predicates Together combining functions, Getting started with the API command-line, Command-Line Editing in ghci, A Simple Command-Line Framework, Reading Command-Line Arguments arguments, reading, Reading Command-Line Arguments editing, Command-Line Editing in ghci commas (,), Lists, Useful Composite Data Types: Lists and Tuples, First Steps with Parsec: Simple CSV Parsing CSV files and, First Steps with Parsec: Simple CSV Parsing tuples, writing, Useful Composite Data Types: Lists and Tuples compact function, Compact Rendering comparison operators, Boolean Logic, Operators, and Value Comparisons, Equality, Ordering, and Comparisons compilers, Your Haskell Environment, Compiling Haskell Source Glasgow Haskell, Your Haskell Environment components (types), Defining a New Data Type composable functors, Thinking More About Functors composite data types, Useful Composite Data Types: Lists and Tuples concat function, More Simple List Manipulations, Left Folds, Laziness, and Space Leaks, The List Monad concurrent programs, Concurrent and Multicore Programming–Conclusions, Hiding Latency, The Main Thread and Waiting for Other Threads–Communicating over Channels, Shared-State Concurrency Is Still Hard–Using Multiple Cores with GHC, A Concurrent Web Link Checker latency, hiding, Hiding Latency main thread waiting for other threads, The Main Thread and Waiting for Other Threads–Communicating over Channels shared-state, Shared-State Concurrency Is Still Hard–Using Multiple Cores with GHC conditional evaluation, Conditional Evaluation–Understanding Evaluation by Example, Conditional Evaluation with Guards constant applicative forms (CAFs), Time Profiling constants, binding C to Haskell, Binding to Constants constraints, Constraints on Type Definitions Are Bad, Constraints on Our Decoder decoding, Constraints on Our Decoder type definitions and, Constraints on Type Definitions Are Bad constructors, Construction and Deconstruction Content-Length field, Parsing Headers continuations, Parsing Headers control-with-character escapes, Control-with-Character Escapes Control.Applicative module, Infix Use of fmap Control.Arrow module, Another Round of Golf control.Concurrent module, Initializing the GUI, Concurrent Programming with Threads concurrent programming with threads, Concurrent Programming with Threads Control.Exception module, The Acquire-Use-Release Cycle, First Steps with Exceptions, Selective Handling of Exceptions Control.Monad module, Another Way of Looking at Monads, Generalized Lifting, Failing Safely with MonadPlus, Writing Tighter Code lifting, Generalized Lifting MonadPlus typeclass and, Failing Safely with MonadPlus Control.Monad.Error module, Usage of the Maybe monad, Monadic use of Either, Error Handling in Monads Control.Monad.Trans module, Designing for Unexpected Uses Control.Parallel module, Transforming Our Code into Parallel Code Control.Parallel.Strategies module, Separating Algorithm from Evaluation Coordinated Universal Time (UTC), ClockTime and CalendarTime cores, Using Multiple Cores with GHC, Understanding Core–Advanced Techniques: Fusion using multiple, Using Multiple Cores with GHC cos function, Numeric Types countEntries function, Stacking Multiple Monad Transformers CSV files, First Steps with Parsec: Simple CSV Parsing–The sepBy and endBy Combinators, The sepBy and endBy Combinators Parsec helper functions and, The sepBy and endBy Combinators ctTZName function, Using CalendarTime ctWDay function, Using CalendarTime ctYDay function, Using CalendarTime currying, using partial functions, Partial Function Application and Currying custom data types for errors, Custom data types for errors c_sin function, Foreign Language Bindings: The Basics D dash (-), as a range character, Filename Matching data keyword, Defining a New Data Type, How to Give a Type a New Identity newtype keyword and, How to Give a Type a New Identity data structures, Defining a New Data Type, The structure, Association Lists–General-Purpose Sequences, Functions Are Data, Too, Taking Advantage of Functions as Data–General-Purpose Sequences functions and, Functions Are Data, Too, Taking Advantage of Functions as Data–General-Purpose Sequences taking advantage of, Taking Advantage of Functions as Data–General-Purpose Sequences data type, defining, Defining a New Data Type–Type Synonyms (see also types) Data.Array module, Introducing Arrays, Folding over Arrays barcode recognition and, Introducing Arrays folding over arrays, Folding over Arrays Data.Bits module, Pretty Printing a String Data.ByteString.Char8 module, Text I/O, The Real Deal: Compiling and Matching Regular Expressions Data.ByteString.Lazy.Char8 module, Text I/O Data.Char module, Transforming Every Piece of Input Data.Dynamic module, Dynamic Exceptions Data.Foldable module, General-Purpose Sequences, Interference with Pure Code Data.Function module, Remembering a Match’s Parity Data.List module, As-patterns, Strictness and Tail Recursion tails function, As-patterns Data.List.lookup function, Association Lists Data.Map module, A Brief Introduction to Maps, Maps–Functions Are Data, Too Data.Monoid module, Lists, Difference Lists, and Monoids Data.Ratio module, Getting Started with ghci, the Interpreter Data.Sequence module, General-Purpose Sequences Data.Traversable module, Interference with Pure Code Data.Typeable module, Dynamic Exceptions database engines, Overview of HDBC Database.HDBC module, Initializing the GUI databases, Using Databases–Error Handling, Connecting to Databases, Simple Queries, Lazy Reading, Database Metadata connecting, Connecting to Databases lazy reading, Lazy Reading metadata, Database Metadata queries, Simple Queries dates, Dates and Times–Extended Example: Piping dates and times, Dates and Times–Extended Example: Piping Daylight Saving Time (DST), ClockTime and CalendarTime -ddump-asm compiler flag, Tuning the Generated Assembly -ddump-simpl compiler flag, Understanding Core, Profile-Driven Performance Tuning deadlocks, Safely Modifying an MVar, Deadlock Dean, Jeffrey, Finding the Most Popular URLs Debian Linux, installing GHC/Haskell libraries, Ubuntu and Debian Linux debugging, Boolean Logic, Operators, and Value Comparisons declarations (module), The Anatomy of a Haskell Module decoding barcodes, Encoding an EAN-13 Barcode deconstructors, Construction and Deconstruction delete function, Getting started with the API DeriveDataTypeable language, Dynamic Exceptions describeTable function, Database Metadata DiffArray type, Modifying Array Elements diffClockTimes function, TimeDiff for ClockTime directories, Directory and File Information disconnect function, Connecting to Databases discriminated unions, The discriminated union div function, Numeric Types do keyword, A Simple Command-Line Framework, Sequencing, Desugaring of do Blocks Monads and, Desugaring of do Blocks sequencing and, Sequencing Doc data type, Generating Test Data doskey command (ghci), Command-Line Editing in ghci double hashing, Turning Two Hashes into Many double quotes ("), writing strings, Strings and Characters, Writing Character and String Literals Double value, Some Common Basic Types, Numeric Types drivers (HDBC), installing, Installing HDBC and Drivers drop function, Functions over Lists and Tuples, Conditional Evaluation dropWhile function, Working with Sublists DST (Daylight Saving Time), ClockTime and CalendarTime duck typing, Static Types dynamic exceptions, Dynamic Exceptions–Error Handling in Monads E EAN-13 barcodes, A Little Bit About Barcodes easyList function, Testing with QuickCheck Either type, Motivation: Boilerplate Avoidance, Use of Either–Exceptions, Monadic use of Either monadic use of, Monadic use of Either elem function, Searching Lists elements function, Generating Test Data ELF object files, Binary I/O and Qualified Imports else keyword, Conditional Evaluation embedded domain specific languages, A Domain-Specific Language for Predicates–Controlling Traversal EmptyDataDecls language extension, Typed Pointers enclose function, Pretty Printing a String endBy function, The sepBy and endBy Combinators #enum construct, Automating the Binding enum keyword (C/C++), The enumeration Enum typeclass, Using CalendarTime enumeration notation, Lists enumeration types, The enumeration environment (programming), Your Haskell Environment environment variables, Environment Variables EOF (end of file), Working with Files and Handles eol function, Lookahead equality tests, The Need for Typeclasses, Equality, Ordering, and Comparisons error function, Handling Errors Through API Design errors, Boolean Logic, Operators, and Value Comparisons, Boolean Logic, Operators, and Value Comparisons, Strong Types, Algebraic Data Types, Reporting Errors, Type Inference Is a Double-Edged Sword, More Helpful Errors, Standard Input, Output, and Error, Handling Errors Through API Design, Reporting Parse Errors, Error Handling, Error Handling–Exceptions, Error Handling with Data Types–Exceptions, Custom data types for errors, Error Handling in Monads, Error Handling API design, handling, Handling Errors Through API Design compiling source code, Type Inference Is a Double-Edged Sword custom data types for, Custom data types for errors handling, Error Handling–Exceptions, Error Handling with Data Types–Exceptions, Error Handling in Monads, Error Handling data types, Error Handling with Data Types–Exceptions databases, Error Handling monads, Error Handling in Monads I/O and, Standard Input, Output, and Error messages, Boolean Logic, Operators, and Value Comparisons, Boolean Logic, Operators, and Value Comparisons, Algebraic Data Types Boolean values and, Boolean Logic, Operators, and Value Comparisons No instance, Boolean Logic, Operators, and Value Comparisons, Algebraic Data Types parsers, handling, Error Handling reporting, Reporting Errors typeclasses, Strong Types, More Helpful Errors ErrorT transformer, Error Handling in Monads escape characters, Strings and Characters escaping text, Escaping Text /etc/passwd file, Extended Example: /etc/passwd–Extended Example: Numeric Types evaluation, Understanding Evaluation by Example–Polymorphism in Haskell, Conditional Evaluation with Guards, Space Leaks and Strict Evaluation–Learning to Use seq conditional with guards, Conditional Evaluation with Guards strict, Space Leaks and Strict Evaluation–Learning to Use seq evaluation strategies, Separating Algorithm from Evaluation event-driven programming, Event-Driven Programming Exception type, First Steps with Exceptions exceptions, Error Handling, Exceptions–Error Handling in Monads, Selective Handling of Exceptions, I/O Exceptions, Throwing Exceptions, Dynamic Exceptions–Error Handling in Monads dynamic, Dynamic Exceptions–Error Handling in Monads I/O (input/output), I/O Exceptions selective handling of, Selective Handling of Exceptions throwing, Throwing Exceptions --exclude flag (hpc), Measuring Test Coverage with HPC executables, creating, Generating a Haskell Program and Importing Modules executeFile function, Using Pipes for Redirection exhaustive patterns, Exhaustive Patterns and Wild Cards explicit recursion, Explicit Recursion exponentiation (**) operator, Undefined Values, and Introducing Variables, Numeric Types exports, The Anatomy of a Haskell Module Exposed-Modules field, Writing a Package Description expressions, Passing an Expression to a Function, Introducing Local Variables functions, passing to, Passing an Expression to a Function let blocks and, Introducing Local Variables external programs, running, Running External Programs extract methods, The Monad Laws and Good Coding Style F fail function, The Monad Typeclass False Boolean value, Boolean Logic, Operators, and Value Comparisons FDs (file descriptors), Using Pipes for Redirection Fedora Linux, installing GHC/Haskell libraries, Fedora Linux fetchAllRowsAL’ function, Lazy Reading fetchAllRows’ function, Reading with Statements FFI (Haskell Foreign Function Interface), Interfacing with C: The FFI–The Real Deal: Compiling and Matching Regular Expressions FFI binding, Compilation Options and Interfacing to C fFlush function, Flushing The Buffer file descriptors (FDs), Using Pipes for Redirection file processing, Efficient File Processing–Putting Our Code to Work filename matching, Filename Matching files, Working with Files and Handles–Extended Example: Functional I/O and Temporary Files, Deleting and Renaming Files, Temporary Files, Efficient File Processing, Filename Matching, Sizing a File Safely–A Domain-Specific Language for Predicates, Directory and File Information, File Modification Times deleting/renaming, Deleting and Renaming Files filename matching, Filename Matching modification times, File Modification Times processing, Efficient File Processing (see file processing) sizing safely, Sizing a File Safely–A Domain-Specific Language for Predicates System.Directory module, using, Directory and File Information temporary, Temporary Files filesystems, I/O Case Study: A Library for Searching the Filesystem–Common Layout Styles searching, I/O Case Study: A Library for Searching the Filesystem–Common Layout Styles filter function, Searching Lists, Selecting Pieces of Input, Filters with interact interact, Filters with interact find command, I/O Case Study: A Library for Searching the Filesystem first function, Another Round of Golf flex, Using Parsec Float type, Numeric Types floating-point numbers, Simple Arithmetic, Lists enumerating, Lists fmap function, Infix Use of fmap, Monads and Functors, Moving Down the Stack monads and, Monads and Functors fold functions, Computing One Answer over a Collection, The Left Fold, Folding from the Right–Left Folds, Laziness, and Space Leaks folding from left, The Left Fold folding from right, Folding from the Right–Left Folds, Laziness, and Space Leaks foldl function, The Left Fold, Folding from the Right–Left Folds, Laziness, and Space Leaks, Left Folds, Laziness, and Space Leaks, Strictness and Tail Recursion foldr function and, Folding from the Right–Left Folds, Laziness, and Space Leaks laziness and space leaks, Left Folds, Laziness, and Space Leaks foldr function, Computing One Answer over a Collection, Folding from the Right–Left Folds, Laziness, and Space Leaks fold’ function, Strictness and Tail Recursion force function, Knowing What to Evaluate in Parallel foreign import declarations, Foreign Language Bindings: The Basics Foreign modules, Foreign Language Bindings: The Basics–Regular Expressions for Haskell: A Binding for PCRE Foreign.C.String module, Foreign Language Bindings: The Basics, Passing String Data Between Haskell and C Foreign.Marshal.Array module, Foreign Language Bindings: The Basics Foreign.Ptr module, Foreign Language Bindings: The Basics ForeignPtr type, Memory Management: Let the Garbage Collector Do the Work forkManaged function, Safe Resource Management: A Good Idea, and Easy Besides forkProcess function, Using Pipes for Redirection forM function, Why Provide Both mapM and forM?

Practical OCaml by Joshua B. Smith

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

cellular automata, Debian, domain-specific language, general-purpose programming language, Grace Hopper, hiring and firing, John Conway, Paul Graham, slashdot, text mining, Turing complete, type inference, web application, Y2K

See CGI comparator function, 96, 137 compare function, 37, 90, 137 comparison functions, 90 compile-time flags, 310 compiler flags, 405 compilers, 405 Complex library, 29 composite types, 68 composition, vs. inheritance, 238 concat function, 93, 97, 137 620Xidxfinal.qxd 9/22/06 4:19 PM Page 447 ■INDEX ■D data member access, 233 data structures, 51, 252 data types, 225 data-driven applications, 2 data-handling functions, for securities trades database, 54–59 databases creating, 51–60 displaying/importing data and, 73–87 reports and, 73–87 DbC (Design by Contract), 133 deadlocks, 313 debuggers, OCaml and, 404 debugging, threads and, 309, 327 default match, 47 delay function, 311, 317 dependency graphs, 145, 148 Design by Contract (DbC), 133 diamond-shaped inheritance, 241 diff command, 380 difftime function, 359 direct objects, 231 directed graphs, 347 directories, reading, 119 -disntr flag, 408 distance calculations, 21 distance type, 41–44 Division_by_zero exception, 127, 172 -dllib –l<LIBNAME> flag, 407 -dllpath <DIR> flag, 407 documentation extracting from comments, 145 ocamldoc for, 145–154 domain-specific languages (DSLs), 203, 411, 415, 419 dot notation, 17 doubles, copying, 352 double_val(v) function, 352 downloads, multipart file, 278 Doxygen, 145 dreaded diamond, 241 DSLs (domain-specific languages), 203, 411, 415, 419 -dtypes compiler flag, 406 duck typing, 231 Dynalink, 289 dynamic linking of code, 356 ■E eager type, 67 EBNF (Extended Backus-Naur Form), 210 echo servers, 179 Eclipse, 14, 402 edit distance, 243–248 Emacs, 14, 402 email type, 136 emptybmp function, 385 encapsulation, 225, 233, 245 encoding, 65 Endianness, 375 End_of_file exception, 78, 126, 131 entry points, 262, 267 Find it faster at http://superindex.apress.com/ concurrency, 271, 309 Condition module, 318 condition variables, 313, 318 configuration file parser (sample), 415–419 Configure macros, 401 constraining types, 35 constraints, parameterized classes and, 234 constructor arguments, 229 contents attribute, 22 conversions, for distances, 31, 41 Conway’s game, 390 Conway, John, 390 cookies, blog server example and, 283–288 Coq proof assistant, 29 correctness, of programs, 271 Cousineau, Guy, 3 cpp (C preprocessor), 411 CPUs, multiple, 310 create function, 317 arrays and, 97 Condition module and, 318 hashtables and, 100 Thread module and, 311, 316 creating arrays, 97 custom exceptions, 127 custom tags/generators, 153 databases, 51–60 enums, 26 functions, 30–32, 33–36 hashtables, 100 http clients, 120 lists, 96 maps, 109 modules, 156 queues, 103 records, 28 servers, 179 sets, 107 sockets, 120 stacks, 105 threads, 310–316 values, 33–36 curried functions, 17, 36, 41 currying functors, 163 custom exceptions, 127 -custom flag, 407, 409 447 620Xidxfinal.qxd 448 9/22/06 4:19 PM Page 448 ■INDEX enums (enumerated values), 23, 26 eprintf function, 75 equality, floats and, 63 error handling/reporting, 18, 137 errorstring exception and, 414 ocamllex and, 201, 205 revised syntax and, 412 errorstring exception, 414 Euclid’s algorithm, 30 event channels, 315, 319 Event module, 315, 319 events, 315–317, 319 Events module, 340 Ex-nunc framework, 276 examples.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 Streams and Parsers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Understanding Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Understanding Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 Example Configuration File Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 Domain-Specific Languages (DSLs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Extending OCaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 More About Quotations and Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 A Longer Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Complex Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 ■CHAPTER 30 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 What This Book Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 ocamllex and ocamlyacc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Camlp4 and Stream Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Where to Go from Here . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 More Functional Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Camlp4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Mailing Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Other Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 Thank You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 Final Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 ■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 xix 620Xfmfinal.qxd 9/22/06 4:21 PM Page xx 620Xfmfinal.qxd 9/22/06 4:21 PM Page xxi About the Author ■JOSHUA B.

OCaml Development Tools (Chapter 28) We talk about OCaml Makefile, findlib, and other tools (including integrated development environments [IDEs]). Here we talk about profiling and debugging, too. This chapter also shows you how to set up your own Ocaml development environment. Camlp4 (Chapter 29) Camlp4, the Ocaml preprocessor and pretty printer, is one of the most powerful and difficultto-understand parts of the Ocaml system. This application and library set enables you to create domain-specific languages (DSLs) on-the-fly and actually rewrite the Ocaml language. This chapter provides a basic understanding of the functionality that Camlp4 provides. Conclusion (Chapter 30) A wrap up of all things OCaml. What Isn’t Covered? This book is more than just a tutorial. I have tried to cover all of the areas of normal OCaml programming. We will not be covering extending the OCaml language in languages other than Ocaml and C.


pages: 226 words: 17,533

Programming Scala: tackle multicore complexity on the JVM by Venkat Subramaniam

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

augmented reality, continuous integration, domain-specific language, don't repeat yourself, loose coupling, semantic web, type inference, web application

The following figure, inspired by Ola Bini’s Language Pyramid (see “Fractal Programming” in Appendix A, on page 211), shows where Scala may fit in with other languages in an enterprise application. An object is said to be immutable if you can’t change its contents once you create it. This eliminates the concerns of managing contention when multiple threads access the object. Java’s String is a great example of an immutable object. 1. Download at Boykma.Com Report erratum Prepared exclusively for sam kaplan this copy is (P1.0 printing, June 2009) 12 W HY S CALA ? Domain-Specific Languages Implemented Using Scala, Groovy, JRuby, ... Manage Concurrency with Scala Intensive Pattern Matching/ Parsing Using Scala Metaprogramming Dynamic Langs like Groovy/JRuby/Clojure Java, Scala, Other Strongly Typed Languages for Infrastructure JVM What about other languages on the JVM...Groovy, JRuby, Clojure? Scala is by far the only prominent strongly typed language that provides functional style and great concurrency support.

You can run it the same way you run programs written using the Java language.5 You can also intermix it well with Java. You can extend Java classes from Scala classes, and vice versa. You can also use Java classes in Scala and Scala classes in Java. You can program applications using multiple languages and be a true Polyglot Programmer6 —you can take advantage of Scala in Java applications where you need concurrency or conciseness (like creating domain-specific languages). Scala is a statically typed language, but, unlike Java, it has sensible static typing. Scala applies type inference in places it can. So, instead of specifying the type repeatedly and redundantly, you can rely on the language to learn the type and enforce it through the rest of the code. You don’t work for the compiler; instead, you let the compiler work for you. For example, when we define var i = 1, Scala immediately figures that the variable i is of type Int.

Yes, Int does not provide us with that method, but that should not stop us from writing such code. We can ask Scala to quietly convert the Int to something that will help us accomplish the previous operation—enter implicit type conversion. Implicit type conversion can help you extend the language to create your own vocabulary or syntax that’s specific to your application and its domain or to create your own domain-specific languages. Let’s start with some crufty code to first understand the concept and then refactor that into a nice class. We need to define the variables ago and from_now and ask Scala to accept the days( ) method. Defining variables is simple, but for it to accept the method, let’s create a class DateHelper that takes an Int as a constructor parameter: import java.util._ class DateHelper(number: Int) { def days(when: String) : Date = { var date = Calendar.getInstance() when match { case "ago" => date.add(Calendar.DAY_OF_MONTH, -number) case "from_now" => date.add(Calendar.DAY_OF_MONTH, number) case _ => date } date.getTime() } } The DateHelper class provides the days( ) method we want.2 Now, all we have to do is convert an Int to a DateHelper.


pages: 554 words: 108,035

Scala in Depth by Tom Kleenex, Joshua Suereth

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

discrete time, domain-specific language, fault tolerance, MVC pattern, sorting algorithm, type inference

: syntax @BeanProperty annotation @Column annotation @cpsParam annotation @ManyToOne annotation @reflect.BeanInfo annotation @reflect.BeanProperty @serializable annotation @specialized annotation @switch annotation, 2nd @tailrec annotation, 2nd, 3rd * method ## method, 2nd #:: method, 2nd + method +: method, 2nd ++ method, 2nd, 3rd, 4th <:< == method, 2nd, 3rd, 4th A abstract class abstract interfaces between software modules reasons for AbstractAddress class AbstractMethodError AccessControlContext accessor, 2nd act method, 2nd, 3rd Actor class, 2nd, 3rd, 4th Actor.actorOf method ActorDispatcher class ActorRef type AdaptiveSearchNode add method addHandle method Address class ALIVE class annotation.target.getter AnnotationHelpers anonfun class, 2nd anonymous classes, serialization of anonymous functions, 2nd AnyRef, 2nd, 3rd, 4th, 5th, 6th AnyVal types App trait Application class applicative functors applicative style, 2nd Applicative.build method ApplicativeBuilder class apply method, 2nd, 3rd, 4th, 5th, 6th, 7th areEqual method ArrayBuffer, 2nd, 3rd ArrayList, 2nd ArraySortTrait avg method B B[A] parameter Bar class, 2nd, 3rd bar method bar.Foo.type Bar.type bar(x) method BeanInfo class benefits, of type classes binary compatibility BinaryFormat type BinaryTree bindings, 2nd BitSet blocks, of code boilerplate BoxesRunTime class boxing Branch class BufferedReader BufferedWriter build method bundleResult method by-name parameters Byte type C C parameter C.super C++ variables Callback type CanBuildFrom class canEqual method case statements, 2nd, 3rd Cat type, 2nd Category Theory and functional programming functors, 2nd monads, 2nd morphism cc target chaining implicits, 2nd changePassword method Child class children method, 2nd class arguments ClassCastExceptions ClassManifest, 2nd closeResource method code blocks, 2nd coding conventions and other languages code blocks Coll type CollectionConverter type collections and methods ArrayBuffer CanBuildFrom hierarchy of immutable collections list stream vector IndexedSeq Iterable iterator LinearSeq List Map mutable collections ArrayBuffer observable synchronization of Observable collections parallel collections, 2nd Seq Set Splitable iterator Stream Traversable TraversableOnce vector views colRank combineResults companion class companion object complexmath, 2nd ComplexNumber class composition inheritance member-composition using constructor arguments CompressedIntSet computeValue concurrency, and immutability conditional execution, using type system heterogeneous typed list IndexedView type Config class, 2nd, 3rd, 4th config file ConnectionDataStore constructor arguments, composition using content method context bounds, 2nd convertToScala count method covariance CreateArray createConnection method createDispatcher method createErrorMessage method createHandle createPreparedStatement method createSearchTree CREATOR class curCount parameter currentIndex method curried method currying, 2nd D dangling operators Data class DataAccess class DatabaseConnection DatabaseResultSetWalker dataId DataStore class DEAD class deadActor default concepts default parameters DefaultHandles delayed construction, of objects delayedInit method DelayedInit trait Dependencies object describeContents method Dog class domain-specific languages. See DSLs. doPrivileged method Double type doubleToReal doWork DriverManager.getConnection method, 2nd, 3rd, 4th DSLs (domain-specific languages) dynamic deoptimization dynamic topology, of actors E eager parsing, in REPL early member definitions EJB (Enterprise JavaBeans) else statements empty implementations, 2nd EmptyList class endofunctor Enterprise JavaBeans. See EJB. Entity beans environment function, 2nd equals method, 2nd escape analysis Event object executeLocalQuery function ExecutorScheduler existential types Expand type experiment-driven development, and REPL Explicit object explicit return statement explicit return types, 2nd expression-oriented programming mutability of no return statements expression-oriented syntax expressions, with parentheses externalbindings.scala F F[_] type, 2nd, 3rd, 4th fa parameter factory method, 2nd failure zones, for actors File parameters, 2nd FileInputStream FileLike, as type classes FileLike.children method FileLineTraversable class, 2nd FileObject FileWrapper class filter method, 2nd, 3rd, 4th, 5th, 6th find method findAnInt method, 2nd first-class functions flatMap method, 2nd, 3rd flatten method, 2nd, 3rd foldLeft method, 2nd, 3rd Foo class, 2nd, 3rd foo method, 2nd, 3rd, 4th, 5th, 6th Foo object, 2nd, 3rd, 4th, 5th Foo type Foo.baz foo.Foo object Foo.type foo.type#Bar Foo#Bar FooHolder fooToBar for expression force method, 2nd, 3rd foreach method, 2nd, 3rd, 4th, 5th ForkJoinPool ForkJoinScheduler forSome method forward method FrontEnd Function interface Function object, 2nd function traits Function1 class functions, in Java functionToPrivilegedAction Functor interface, 2nd Functor.apply method Functor.map method functorOps functors, 2nd Future object, 2nd G GathererNode, 2nd Generic types GenericSortTrait GenIterator GenSeq GenTraversableOnce get method, 2nd, 3rd, 4th getConnection function getFoo getLines method getNextChild method getOrElse method getstatic operation getTemporaryDirectory method getValue Google Collections, concepts of functional programming in H Handle type, 2nd, 3rd handleMessage method hashCode method, 2nd HashMaps, 2nd HashSet HasLogger hasNext method HCons class, 2nd head method, 2nd, 3rd HeadNode, 2nd, 3rd, 4th, 5th heterogeneous typed list, 2nd hierarchy, of collections higher-kinded types HList class, 2nd HListViewN class, 2nd HNil class, 2nd holder object HotSpot runtime optimizer HttpSession I i object, 2nd identifiers if block if clause if statements, 2nd, 3rd If type if_icmpne imaginary method, 2nd immutability and concurrency and object equality immutable collections list stream vector immutable references ImmutableHashMap ImmutableService Imperative coding implicit constraints, 2nd implicit conversions and object identity chaining implicits implicit method, 2nd, 3rd implicit parameters, with defaults implicit views, 2nd implicitly function import statement, 2nd index service IndexedSeq class, 2nd IndexedView type indexN function inexpressible language features, in REPL inheritance, composition including, 2nd initCoreSize inlining inner type InputChannel InputStream, 2nd InputStreamReader insert method InstantaneousTime class int type interfaces, abstract between software modules reasons for IntHolder class IntStore class intToString InvalidClassException IScheduler interface isDirectory isEmpty method isLoggedIn method ItemType Iterable interface, 2nd, 3rd, 4th IterableLike Iterables object Iterator method, 2nd J Java and Scala annotations and static fields annotation targets benefits of JVM implicit conversions and object identity chaining implicits language differences in primitive boxing in visibility unique features Scala objects in serialization Java class Java interfaces java.awt.Component java.io.File, 2nd java.lang.Class java.lang.IndexOutOfBounds-Exception java.lang.Integer type, 2nd java.lang.Object, 2nd, 3rd java.lang.String type java.net.URL java.security class java.sql.DriverManager.get-Connection(...) method java.util.ArrayList, 2nd java.util.Collections class java.util.concurrent.Executor java.util.concurrent.Executors java.util.Date class java.util.Executor java.util.List, 2nd JavaClass JavaConversions javap, 2nd javax.swing.JComponent jdbc_password JdbcTemplate class JdbcTemplate method jdbc_url jdbc_user JRebel JVM bytecode JVM, benefits of K KittyDoggy class L lambdas language differences, Java and Scala in primitive boxing in visibility unique features LazyTraversable type Leaf type LeafNode, 2nd limiting scope, of implicits creating implicits for import without requiring import LinearSeq, 2nd, 3rd lineLengthCount method link method, 2nd List class, 2nd, 3rd, 4th list collections List(start) method ListView loan method loaner pattern LoggedDataAccess class Logger class login method logout method longWrapper lookUp method loop method M makeLineTraversable method makeList method ManagedResource ManagedResource.writeFile method Manifest class MatchError MatrixService.multiply maxDocs maxResponses maxResults maxSize member-composition, 2nd menu button click MessageDispatcher method inlining method parameters MixableParent mkdirs method MODULE$, 2nd Monad type, 2nd monadOps Monads, 2nd move method multiple inheritance, of objects mutability, of expression-oriented programming mutable collections ArrayBuffer observable synchronization of Mutable objects MutableService mutation statements N NaiveQuickSort object naiveWrap method named and default parameters named parameters, 2nd naming, variables Nat type nesting, scope of implicits via NetworkEntity, 2nd newMethod method next method, 2nd, 3rd, 4th NextIdxView NilTree Node class NoManifest class None creating new object or returning default executing block of code if variable is initialized using potential variables to construct another NonZero type null object NullDispatcher Numeric type NumericRange O object equality, and immutability object identity, and implicit conversions ObjectInputStream ObjectOutputStream observable collections ObservableBuffer ObservableMap ObservableSet observe method On Stack Replacement operator notation operator overloading operators, dangling optimization tableswitch optimization tail recursion optimization Option class, 2nd, 3rd, 4th, 5th OptManifest or method Ordering type, 2nd, 3rd OriginalType OtherItemType Outer class OutputChannel, 2nd OutputStream overridden methods override keyword, 2nd P package.scala par method parallel collections, 2nd parameters, 2nd parameterspaths, and types ParArray Parent class parentheses, expressions with ParentNode parsedConfigFile method parsing, in REPL ParVector paths, and types, 2nd pattern matching, 2nd, 3rd Player class PlayerStatus plus method Point2 class Point2D class polymorphic equality implementing timeline library example, 2nd postfix operator Predef class Predicate interface Predicates class PreparedStatementCreator interface Prev type primitive boxing, in Java and Scala primitive widenings primitives println method, 2nd, 3rd, 4th private keyword private variables PrivilegedAction PrivilegedExceptionAction Property trait PureAbstract Q qsort QueryResponse QuickSort, 2nd Quicksort method QuickSortBetterTypes object R randomElement method Range object react method, 2nd Read Eval Print Loop.

Because a lot of developers don’t have strong type theory backgrounds, this often requires more experimentation. Experiment-driven development is about experimenting in the REPL with the type system to utilize types as effectively as possible in your API. Experiment-driven design is more about adding larger features or domains into your code, rather than new methods or bug fixes. Experiment-driven design can also help drastically when defining domain-specific languages (DSLs). A DSL is a pseudo programming language that deals with a particular domain. This language is specific to the domain at hand—for example, querying for data from a database. A DSL may be either internal, as seen in many Scala libraries, or external like SQL. In Scala, it is popular among library developers to create DSLs covering the same domain as the library. For example, the Scala actors library defines a DSL for sending and receiving messages in a thread-safe manner.

Defining types from class, trait, or object keywords As seen in the example, class and trait names can be referenced directly when annotating types within Scala. When referring to an object’s type, you need to use the type member of the object. This syntax isn’t normally seen in Scala, because if you know an object’s type, you can just as easily access the object directly, rather than ask for it in a parameter. Using objects as parameters Using objects as parameters can greatly help when defining domain specific languages, as you can embed words as objects that become parameters. For example, we could define a simulation DSL as follows: object Now object simulate { def once(behavior : () => Unit) = new { def right(now : Now.type) : Unit = ... } } simulate once { () => someAction() } right Now 6.1.1. Types and paths Types within Scala are referenced relative to a binding or path. As discussed in chapter 5, a binding is the name used to refer to an entity.


pages: 357 words: 63,071

Essential SQLAlchemy by Rick Copeland

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

database schema, defense in depth, domain-specific language, web application

(question mark) as a name value, SQL Expression Language | bitwise logical operator, Operators and functions in WHERE clauses ~ bitwise logical operator, Operators and functions in WHERE clauses A active defaults, Defaults “active record” patterns, The data mapper pattern acts_as_versioned( ) statement, Versioned Extension adds( ) decorator, Using custom collections in relations add_column( ) method, Customizing the Select Statement in ORM Queries, Other Query Methods add_entity( ) method, Customizing the Select Statement in ORM Queries, Other Query Methods after_commit( ) (SessionExtension class), Extending Sessions after_delete( ) hook (MapperExtension), Extending Mappers after_flush( ) (SessionExtension class), Extending Sessions after_flush_postexec( ) (SessionExtension class), Extending Sessions after_insert( ) hook (MapperExtension), Extending Mappers @after_revert decorator, Versioned Extension after_update( ) hook (MapperExtension), Extending Mappers alias( ) method, Using aliases aliases, Using aliases all parameter, Cascading Changes to Related Objects all( ) method, Querying at the ORM Level, Other Query Methods allow_column_override parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters allow_null_pks parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters ALTER TABLE command, Foreign keys always_refresh parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters AND boolean operator, Operators and functions in WHERE clauses any( ) method, Querying at the ORM Level append( ) function, Updating Objects in the Session appender( ) decorator, Using custom collections in relations append_result( ) hook (MapperExtension), Extending Mappers application-specific custom types, Application-Specific Custom Types apply_avg( ) method, Other Query Methods apply_labels( ) method, Using the “generative” query interface apply_max( ) method, Other Query Methods apply_min( ) method, Other Query Methods apply_sum( ) method, Other Query Methods arbitrary selectables, mapping, Mapping Arbitrary Selectables architecture (SQLAlchemy), SQLAlchemy Architecture, Object Relational Mapper (ORM) *args argument Column constructor, Column Definitions Table constructor, Defining Tables arithmetic operators, Operators and functions in WHERE clauses asc( ) method, Querying at the ORM Level AssertionPool pool type (sqlalchemy.pool), Connection Pooling associable( ) function, Associable Extension association proxys, Association Proxy as_scalar( ) method, Using the “generative” query interface attribute Elixir keyword argument, Fields and Properties attribute-based syntax (Elixir), Attribute-based syntax attribute_mapped_collection( ) method, Using custom collections in relations autoflush argument (sessionmaker( ) method), Creating a Session autoflush feature, flushing sessions, The unit of work pattern autoflush( ) method, Other Query Methods autoincrement argument (Column constructor), Column Definitions autoload parameter, MetaData Management Table object, Defining Tables autoload_with argument (Table), Defining Tables AVG( ) function, Other Query Methods avg( ) method, Other Query Methods B backref( ) function, Using BackRefs, Cascading Changes to Related Objects parameters, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters backrefs, Using BackRefs batch parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters before_commit( ) (SessionExtension class), Extending Sessions before_delete( ) hook (MapperExtension), Extending Mappers before_flush( ) (SessionExtension class), Extending Sessions before_insert( ) hook (MapperExtension), Extending Mappers before_update( ) hook (MapperExtension), Extending Mappers begin( ) method, Flushing, Committing, and Rolling Back Session Changes belongs_to( ) statement, DSL syntax between( ) method, Querying at the ORM Level ClauseElement, Operators and functions in WHERE clauses bind argument MetaData.reflect( ) method, Table reflection sessionmaker( ) method, Creating a Session bind parameters, Getting Started with MetaData, Operators and functions in WHERE clauses custom, using, Using custom bind parameters MetaData methods and, Create/drop MetaData and schema objects select( ), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method bind( ) method, Using SqlSoup for ORM-Style Queries and Updates bind=None parameter (text( ) function), Using literal text in queries bindparam( ) method, Using custom bind parameters, Using literal text in queries binds argument (sessionmaker( ) method), Creating a Session boolean operators, Operators and functions in WHERE clauses bound MetaData, Binding MetaData bounded constructors, Getting Started with MetaData built-in type TypeEngine objects, Built-in Types Byer, Mike, What Is SQLAlchemy C c attribute (mapped class), SQLAlchemy Philosophy .c objects, SQL Expression Language cascade parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters CheckConstraint, CHECK constraints checkfirst parameter (MetaData), Create/drop MetaData and schema objects CheeseShop, Installing setup tools ClauseElement objects, Operators and functions in WHERE clauses clear( ) method, Using SqlSoup for ORM-Style Queries and Updates clear_mappers( ) function, Basic Object Mapping close( ) method, Result set objects, Other Session Methods ResultProxy object, Database Connections and ResultProxys collection_class parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters, Using custom collections in relations colname Elixir keyword argument, Fields and Properties colname parameter (ManyToOne), Attribute-based syntax Column class, Defining Tables definitions, Column Definitions column parameter (ForeignKey constructor), Foreign keys column( ) method, Using the “generative” query interface columns parameter (ForeignKeyConstraint class), Foreign keys columns=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method column_kwargs parameter (ManyToOne), Attribute-based syntax column_mapped_collections( ) method, Using custom collections in relations column_prefix keyword, Customizing the name of the mapped column column_prefix parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters column_property( ) function, Mapping subqueries, Elixir deferred properties commit( ) method, Flushing, Committing, and Rolling Back Session Changes compare_with( ) method, Versioned Extension comparison operators, Operators and functions in WHERE clauses __composite_values__( ) method, Mapping composite values concat( ) method, Querying at the ORM Level concrete parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters concrete table inheritance, Overview of Inheritance Mapping mapping, Concrete Table Inheritance Mapping, Joined Table Inheritance Mapping configure( ) method (Session), Creating a Session connect( ) method, Connection pooling, Database Connections and ResultProxys, Connection Pooling Connection object, Database Connections and ResultProxys connection pools, Connection pooling, Engines and MetaData, Connection Pooling, MetaData Engine class and, Engines and Connectables connect_args parameter (create_engine( ) function), Engines and Connectables Constraint class, Constraints, Defaults constraint parameter (ForeignKey constructor), Foreign keys constraint_kwargs parameter (ManyToOne), Attribute-based syntax contains( ) method, Querying at the ORM Level contains_alias( ) method, ORM Querying with Joins contains_eager( ) method, ORM Querying with Joins, Customizing the Select Statement in ORM Queries contextual sessions, Contextual or Thread-Local Sessions convert_bind_param( ) method, Implementing a TypeDecorator convert_result_value( ), Implementing a TypeDecorator convert_unicode parameter (create_engine( ) function), Engines and Connectables correlate( ) method, Using the “generative” query interface, Correlated versus uncorrelated subqueries correlate=True parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method COUNT( ) function, Other Query Methods count( ) method, Using the “generative” query interface, Other Query Methods CREATE TABLE statement, Foreign keys create( ) method, Defining Tables, Type System Overview MetaData/schema objects and, Create/drop MetaData and schema objects create_all( ) method, Create/drop MetaData and schema objects, Type System Overview create_engine( ) function, Engine, Engines and Connectables configuring logging and, Configuring SQLAlchemy Logging create_instance( ) hook (MapperExtension), Extending Mappers creator parameter (create_engine( ) function), Engines and Connectables D data manipulation language (DML), Inserts, Updates, and Deletes data mapper pattern, Design Concepts in the ORM database drivers, installing, Installing Some Database Drivers datetime.now method, Active defaults DB-API interface, What Is SQLAlchemy DB2 database, What Is SQLAlchemy decorators, Using custom collections in relations default argument (Column constructor), Column Definitions insert defaults and, Active defaults defaults, Defaults, Defining Indexes defer( ) function, Eager versus deferred loading defer( ) method, ORM Querying with Joins deferred Elixir keyword argument, Fields and Properties deferred( ) function, Eager versus deferred loading delete parameter, Cascading Changes to Related Objects DELETE statements, Delete Statements delete( ) function, Performing Queries and Updates SqlSoup, using, Using SqlSoup for ORM-Style Queries and Updates delete( ) method, Delete Statements, Other Session Methods, Querying Using Elixir, Using SqlSoup for ORM-Style Queries and Updates sessions, deleting objects from, Deleting Objects from the Session delete-orphan, Cascading Changes to Related Objects desc( ) method, Querying at the ORM Level Detached state (Session), Saving Objects to the Session dialect management (SQL), SQL dialect management dialect-specific types, Dialect-Specific Types distinct( ) method, Ordering and grouping results, returning distinct values, Using the “generative” query interface, Querying at the ORM Level, Other Query Methods ClauseElement object, Operators and functions in WHERE clauses distinct=False parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method DML (data manipulation language), Inserts, Updates, and Deletes domain specific language (DSL) syntax, Using Elixir, DSL syntax drivers, installing, Installing Some Database Drivers drop_all( ) method, Create/drop MetaData and schema objects DSL (domain specific language) syntax, Using Elixir, DSL syntax dynamic_loader( ) method, Updating Objects in the Session E eagerload( ) method, ORM Querying with Joins eagerload_all( ) method, ORM Querying with Joins Easyinstall, Getting Started easy_install tools, Installing setup tools echo argument configuring logging and, Configuring SQLAlchemy Logging echo argument (create_engine( ) function), Engines and Connectables echo_pool argument configuring logging and, Configuring SQLAlchemy Logging create_engine( ) function), Engines and Connectables echo_uow argument (sessionmaker( ) method), Creating a Session echo_uow flag (create_engine( ) function), Configuring SQLAlchemy Logging EGG files, Installing setup tools Elixir, Elixir: A Declarative Extension to SQLAlchemy, Versioned Extension extensions, Elixir Extensions, Versioned Extension inheritance, Inheritance installing/using, Installing Elixir querying using, Querying Using Elixir relations, Relations elixir.ext.encrypted extension, Encrypted Extension elixir.ext.versioned extension, Versioned Extension __emulates__ class attribute, Using custom collections in relations encoding parameter (create_engine( ) function), Engines and Connectables endswitch( ) method, Querying at the ORM Level endswith( ) method (ClauseElement), Operators and functions in WHERE clauses Engine class, Engine, MetaData Management, Object Relational Mapper (ORM), Engines and MetaData, Adapt Tables from one MetaData to another connection pools, Connection pooling managing connections, Engines and Connectables, MetaData Engine pools connection pools, Connection Pooling entities, The Object/Relational “Impedance Mismatch” entity_name parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters __eq__( ) method, Mapping composite values EXCEPT clause, Set operations (UNION, INTERSECT, EXCEPT) except_( ) function, Set operations (UNION, INTERSECT, EXCEPT) except_( ) method, Using the “generative” query interface except_all( ) function, Set operations (UNION, INTERSECT, EXCEPT) except_all( ) method, Using the “generative” query interface exclude_properties, Using include_properties and exclude_properties exclude_properties parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters execute( ) method, Database Connections and ResultProxys, Inserts, Updates, and Deletes, Other Session Methods executemany( ) method, Insert Statements expire( ) method, Other Session Methods, Querying Using Elixir explicit execution, Getting Started with MetaData explicit sequences, creating, Creating Explicit Sequences expression language (SQL), SQL Expression Language, Object Relational Mapper (ORM) expression-builders (Pythonic), What Is SQLAlchemy expunge parameter, Cascading Changes to Related Objects expunge( ) method, Other Session Methods, Querying Using Elixir extending session, Extending Sessions extension argument (sessionmaker( ) method), Creating a Session extension parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters, Extending Mappers extension( ) method, ORM Querying with Joins ex_setup.py file, Installing setup tools F fetchall( ) method, Result set objects ResultProxy), Database Connections and ResultProxys fetchmany( ) method, Result set objects fetchone( ) method, Result set objects ResultProxy, Database Connections and ResultProxys Field( ) class, Fields and Properties filter( ) method, Mapping Objects to Tables, Querying at the ORM Level, Other Query Methods querying with joins, ORM Querying with Joins filter_by( ) method, Object Relational Mapper (ORM), Mapping Objects to Tables, Querying at the ORM Level, Other Query Methods, Querying Using Elixir querying with joins, ORM Querying with Joins Firebird database, Other supported drivers first( ) (Query object), Querying at the ORM Level first( ) method, Other Query Methods flush( ) function, Mapping Objects to Tables saving objects to sessions and, Saving Objects to the Session flush( ) method, Flushing, Committing, and Rolling Back Session Changes, Querying Using Elixir, Using SqlSoup for ORM-Style Queries and Updates for+update=False parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method foreign keys, The Object/Relational “Impedance Mismatch” ForeignKey constructor, Foreign keys ForeignKeyConstraint class, Foreign keys foreign_keys parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters for_fields parameter, Encrypted Extension for_update parameter (Sequence constructor), Creating Explicit Sequences from_obj=[ ] parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method, Subqueries from_statement( ) method, Customizing the Select Statement in ORM Queries, Other Query Methods G “generative” interface, Using the “generative” query interface get( ) (Query object), Querying at the ORM Level get( ) hook (MapperExtension), Extending Mappers get( ) method, Mapping Objects to Tables, Other Session Methods, Other Query Methods, Querying Using Elixir Elixir, querying using and, Querying Using Elixir __getattr__( ) method, Using SqlSoup for ORM-Style Queries and Updates __get attr__( ) method (ResultProxy), Database Connections and ResultProxys __getitem__( ) method, Other Query Methods get_as_of( ) method, Versioned Extension get_by( ) method, Querying Using Elixir get_indent( ) function, Contextual or Thread-Local Sessions get_session( ) hook (MapperExtension), Extending Mappers GROUP BY clause, Ordering and grouping results, returning distinct values group_by( ) method, Using the “generative” query interface, Other Query Methods group_by=Name parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method H hand-generated SQL versus SQLAlchemy generation layers, SQL Expression Language has( ) method, Querying at the ORM Level has_and_belongs_to_many( ) statement, DSL syntax has_field( ) method, Fields and Properties has_field( ) statement, DSL syntax has_many( ) method, DSL syntax has_one( ) statement, DSL syntax HAVING clause, Ordering and grouping results, returning distinct values, Other Query Methods having( ) method, Using the “generative” query interface, Other Query Methods having=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method horizontal partitioning, Horizontal Partitioning I identity_map attribute, Other Session Methods idiomatically Python (Pythonic), What Is SQLAlchemy IF EXISTS clause, Create/drop MetaData and schema objects IF NOT EXISTS clause, Create/drop MetaData and schema objects “impedance mismatch” (object/relational), The Object/Relational “Impedance Mismatch” implicit execution, Getting Started with MetaData IN clause, Embedding subqueries in an IN clause include_columns argument (Table), Defining Tables include_properties parameter, Using include_properties and exclude_properties mapper( ) function, Other mapper⁠(⁠ ⁠) Parameters increment parameter (Sequence constructor), Creating Explicit Sequences index argument (Column constructor), Column Definitions Index object, The Index object indexes, Defining Indexes Informix database, Other supported drivers ingerits parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters inheritance mapping, Inheritance Mapping, Relations and Inheritance concrete table, Concrete Table Inheritance Mapping, Joined Table Inheritance Mapping joined table, Joined Table Inheritance Mapping, Relations and Inheritance inherit_foreign_keys parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters inhert_condition parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters init_failed( ) hook (MapperExtension), Extending Mappers init_instance( ) hook (MapperExtension), Extending Mappers injection attacks (SQL), What Is SQLAlchemy insert defaults, Active defaults INSERT statements, Performing Queries and Updates, Insert Statements insert( ) function, Insert Statements SqlSoup, using, Using SqlSoup for ORM-Style Queries and Updates installation (SQLAlchemy), Installing SQLAlchemy, SQLAlchemy Tutorial instances( ) method, Other Query Methods instrument_class( ) hook (MapperExtension), Extending Mappers internally_instrumented( ) decorator, Using custom collections in relations INTERSECT clause, Set operations (UNION, INTERSECT, EXCEPT) intersect( ) function, Set operations (UNION, INTERSECT, EXCEPT) intersect( ) method, Using the “generative” query interface intersect_all( ) method, Using the “generative” query interface inverse parameter, Attribute-based syntax OneToMany), Attribute-based syntax in_( ) method, Querying at the ORM Level ClauseElement, Operators and functions in WHERE clauses, Subqueries items( ) method (ResultProxy), Database Connections and ResultProxys iterator( ) decorator, Using custom collections in relations __iter__( ) method, Result set objects, Other Query Methods ResultProxy), Database Connections and ResultProxys J join( ) function, Object Relational Mapper (ORM) join( ) method, Using the “generative” query interface, Joining selectables, Other Query Methods, Using SqlSoup for ORM-Style Queries and Updates SqlSoup and, Joins with SqlSoup joined table inheritance, Overview of Inheritance Mapping joined table inheritance mapping, Joined Table Inheritance Mapping, Relations and Inheritance join_depth parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters K key argument (Column constructor), Column Definitions key method (ResultProxy), Database Connections and ResultProxys keys( ) method (ResultProxy), Database Connections and ResultProxys L label( ) method, Using the “generative” query interface ClauseElement), Operators and functions in WHERE clauses lazy parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters, Associable Extension lazyload( ) method, ORM Querying with Joins LEFT OUTER JOIN, Customizing the Select Statement in ORM Queries library logging, Configuring SQLAlchemy Logging like( ) method, Querying at the ORM Level ClauseElement, Operators and functions in WHERE clauses LIMIT clause, Limiting results returned LIMIT modifier, Other Query Methods limit( ) method, Using the “generative” query interface, Other Query Methods limit=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method literal text in queries, Using literal text in queries load( ) hook (MapperExtension), Extending Mappers load( ) method, Other Session Methods, Other Query Methods local_side parameter (ManyToMany), Attribute-based syntax M M:N relationships, Basic Relationships, 1:1 relations backrefs and, Using BackRefs many-to-many (M:N) relationships, M:N relations backrefs, Using BackRefs ManyToMany object, Attribute-based syntax ManyToOne object, Attribute-based syntax map( ) method, Using SqlSoup for ORM-Style Queries and Updates mapped classes, SQLAlchemy Philosophy MappedProduct class, Using SqlSoup for ORM-Style Queries and Updates mapped_collection( ) method, Using custom collections in relations mapper patterns, The data mapper pattern mapper( ) function, Basic Object Mapping, ORM Querying with Joins extending mappers and, Extending Mappers inheritance hierarchy and, Single Table Inheritance Mapping parameters, Other mapper⁠(⁠ ⁠) Parameters MapperExtension class, Extending Mappers, Extending Sessions MAX( ) function, Other Query Methods max( ) method, Other Query Methods max_overflow parameter (create_engine( ) function), Engines and Connectables merge parameter, Cascading Changes to Related Objects merge( ) method, Other Session Methods, Querying Using Elixir metadata argument Table constructor, Defining Tables MetaData class, Engine, MetaData Management, Engines and MetaData, MetaData, Adapt Tables from one MetaData to another connecting to databases/tables, Connecting to the Database and Creating Some Tables object mappers, declaring, Declaring Object Mappers ORM and, Object Relational Mapper (ORM) tables, defining, Defining Tables, Column Definitions TypeEngine objects and, Type System Overview metadata.bind.echo property, Performing Queries and Updates MIN( ) function, Other Query Methods min( ) method, Other Query Methods module parameter (create_engine( ) function), Engines and Connectables mustexist argument (Table), Defining Tables MySQL, What Is SQLAlchemy drivers, installing, Other supported drivers N name argument Column constructor, Column Definitions Table constructor, Defining Tables name parameter ForeignKey constructor, Foreign keys Sequence constructor, Creating Explicit Sequences name parameter (Constraint class), Constraints “named colon” format, Using literal text in queries nested loops, The Object/Relational “Impedance Mismatch” __ne__( ) method, Mapping composite values noload( ) method, ORM Querying with Joins nonpolymorphic inheritance, Relations and Inheritance non_primary parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters NOT boolean operator, Operators and functions in WHERE clauses nullable argument (Column constructor), Column Definitions NullPool pool type (sqlalchemy.pool), Connection Pooling O object-relational mapper (see ORM) object/relational “impedance mismatch”, The Object/Relational “Impedance Mismatch”, SQLAlchemy Philosophy objects tables, mapping to, Mapping Objects to Tables OFFSET clause, Limiting results returned offset( ) method, Using the “generative” query interface, Other Query Methods offset=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method “oid” (object ID), Engines and Connectables ON clause, Joining selectables ON DELETE CASCADE statement, Cascading Changes to Related Objects ON UPDATE clause, Foreign keys ondelete parameter, Cascading Changes to Related Objects ForeignKey constructor, Foreign keys ondelete parameter (ManyToOne), Attribute-based syntax one( ) method, Other Query Methods Query object, Querying at the ORM Level OneToMany object, Attribute-based syntax OneToOne object, Attribute-based syntax only argument (MetaData.reflect( ) method), Table reflection onupdate argument Column constructor, Column Definitions onupdate parameter ForeignKey constructor, Foreign keys onupdate parameter (ManyToOne), Attribute-based syntax on_link( ) decorator, Using custom collections in relations op( ) method (ClauseElement), Operators and functions in WHERE clauses option( ) method, Extending Mappers optional parameter (Sequence constructor), Creating Explicit Sequences options( ) method, ORM Querying with Joins, Other Query Methods OR boolean operator, Operators and functions in WHERE clauses Oracle, What Is SQLAlchemy drivers, installing, Other supported drivers ORDER BY clause, Ordering and grouping results, returning distinct values ordered collections, Ordering List ordering_list( ) method, Ordering List order_by parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters ManyToMany), Attribute-based syntax mapper( ) function, Other mapper⁠(⁠ ⁠) Parameters OneToMany), Attribute-based syntax order_by( ) method, Using the “generative” query interface, Other Query Methods order_by=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method ORM (object-relation mapper) extending mappers, Extending Mappers, ORM Partitioning Strategies ORM (object-relational mapper), What Is SQLAlchemy, Object Relational Mapper (ORM), Object Relational Mapper (ORM) (see also mapper) declaring, Declaring Object Mappers, Declaring Relationships Between Mappers design concepts in, Design Concepts in the ORM MetaData object and, MetaData Operations partitioning strategies, ORM Partitioning Strategies property mapping, customizing, Customizing Property Mapping querying, Querying at the ORM Level, Contextual or Thread-Local Sessions querying and updating, Querying and Updating at the ORM Level, Using Contextual Sessions with Mappers and Classes relationships, declaring, Declaring Relationships Between Mappers, Extending Mappers self-referential mappers, using, Using a Self-Referential Mapper ORM object-relational mapper, Building an Object Mapper, Horizontal Partitioning outerjoin( ) method, Using the “generative” query interface, Joining selectables, Other Query Methods owner argument (Table), Defining Tables P params( ) method, Other Query Methods passive defaults, Passive defaults PassiveDefault instances, Passive defaults passive_deletes parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters Pending objects, Updating Objects in the Session Pending state (Session), Saving Objects to the Session Persistent object, Updating Objects in the Session Persistent state (Session), Saving Objects to the Session PIL (Python Imaging Library), Types System “plain old Python objects” (see POPOs) plural_name argument (associable( ) function), Associable Extension polymorphic class, Relations and Inheritance identity of rows, Single Table Inheritance Mapping polymorphic_fetch parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters polymorphic_identity argument, Single Table Inheritance Mapping mapper( ) function, Other mapper⁠(⁠ ⁠) Parameters polymorphic_on argument, Single Table Inheritance Mapping polymorphic_on parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters polymorphic_union( ) function, Concrete Table Inheritance Mapping pool parameter (create_engine( ) function), Engines and Connectables pool.manage( ) function, Connection Pooling pool.manage( ) method, Connection Pooling poolclass parameter (create_engine( ) function), Engines and Connectables pools (connections), Connection pooling (see also connection pools) pool_recycle parameter (create_engine( ) function), Engines and Connectables pool_size parameter (create_engine( ) function), Engines and Connectables pool_timeout parameter (create_engine( ) function), Engines and Connectables POPOs (plain old Python objects), What Is SQLAlchemy populate_existing( ) method, Other Query Methods populate_instance( ) hook (MapperExtension), Extending Mappers PostgreSQL, What Is SQLAlchemy drivers, installing, Other supported drivers passive defaults, support for, Passive defaults post_update parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters prefixes=None parameter (select( )), The select⁠(⁠ ⁠) function versus the select⁠(⁠ ⁠) method prefix_with( ) method, Using the “generative” query interface PrimaryDeyConstraint object, Primary keys primaryjoin parameter, Other relation⁠(⁠ ⁠) and backref⁠(⁠ ⁠) Parameters primary_key argument (Column constructor), Column Definitions primary_key parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters properties parameter (mapper( ) function), Other mapper⁠(⁠ ⁠) Parameters property mapping, Customizing Property Mapping PyPI (Python package index), Installing setup tools pysqlite binary module, Installing the SQLite driver on Python versions before 2.5 pysqlite driver, SQLAlchemy Tutorial Python Imaging Library (PIL), Types System Pythonic (idiomatically Python), What Is SQLAlchemy Q queries, Performing Queries and Updates, Running Queries and Updates constructing, Basic Query Construction, Joins and Set Operations query( ) method, Other Session Methods, Querying at the ORM Level query_from_parent( ) method, Other Query Methods question mark (?)

Assuming you have already installed SetupTools and SQLAlchemy as described in Chapter 2, you can install Elixir on Unix-like systems—including Linux, BSD, and OS X—as follows: $ sudo easy_install -UZ Elixir On Windows, the command is similar: c:\>easy_install -UZ Elixir To verify that Elixir is installed properly, open up an interactive Python interpreter, import the module, and verify its version: >>> import elixir >>> elixir.__version__ '0.4.0' And that’s all there is to it. Elixir is installed! Using Elixir Elixir has two syntaxes for defining your classes: an attribute-based syntax (shown previously) and a “domain specific language” (DSL) syntax. Both have similar power; which one you use is mostly a matter of personal style. The DSL-based syntax may be phased out in the future, as it is no longer the “default” syntax, but it is not currently deprecated, so it is covered in this chapter. If we were to define the product database using the DSL syntax, for instance, we would write the following (with the methods for each class omitted for clarity): from elixir import * metadata.bind = 'sqlite://' class Product(Entity): has_field('sku', String(20), primary_key=True) has_field('msrp', Numeric) has_many('prices', of_kind='Price') class Store(Entity): has_field('name', Unicode(255)) has_many('prices', of_kind='Price') class Price(Entity): has_field('price', Numeric, default=0) belongs_to('product', of_kind='Product') belongs_to('store', of_kind='Store') There is a rough correspondence between the functionality of the attribute-based syntax for defining entities and the DSL syntax.


pages: 46 words: 6,267

Managing Infrastructure With Puppet by James Loope

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

domain-specific language

It then decides on a set of changes that need to occur to bring the catalog state into agreement with your descriptions. The execution is idempotent, meaning that only the changes needed to bring the state into agreement with the description will be made. The entire catalog can be run over and over again without causing deviation from the described state. These resource descriptions are made in a Domain Specific Language implemented in Ruby. This means that the syntax is often similar to Ruby, but you cannot simply write Ruby code in a Puppet manifest and have it executed. In fact, the language is declarative, rather than imperative like Ruby. With Puppet, you say how you want things to look, as opposed to describing what should be done to make them look that way. It’s Puppet’s job to know how to make that description reality.


pages: 713 words: 93,944

Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement by Eric Redmond, Jim Wilson, Jim R. Wilson

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, create, read, update, delete, data is the new oil, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, full text search, general-purpose programming language, linked data, MVC pattern, natural language processing, node package manager, random walk, recommendation engine, Ruby on Rails, Skype, social graph, web application

A graph of nodes using a custom profile Although the web interface is an easy way to make a few edits, we need a more powerful interface for production work. Neo4j via Gremlin There are several languages that interoperate with Neo4j: Java code, REST, Cypher, Ruby console, and others. The one we’ll use today is called Gremlin, which is a graph traversal language written in the Groovy programming language. You needn’t actually know Groovy to use Gremlin, however, so think of it as just another declarative domain-specific language, like SQL. Like other consoles we’ve explored, Gremlin provides access to the underlying language infrastructure on which it’s based. This means you can use Groovy constructs and Java libraries in Gremlin. We found it a powerful and more natural way of interacting with graphs than Neo4j’s native Java code. And even better, the Gremlin console is available in the Web Admin; just click the Console link at the top, and choose Gremlin.

g.V.both('friends').dedup.transform{​​ ​​ [ it.name, it.out('likes').name.toList() ]​​ ​​}​​ ​​==> [Alice, [Prancing Wolf Ice Wine 2007]]​​ ​​==> [Patty, []]​​ ​​==> [Tom, [Prancing Wolf Ice Wine 2007, Prancing Wolf Kabinett 2002]]​​ Gremlin definitely takes a little getting used to, especially if you haven’t done much Groovy programming before. Once you get the hang of it, you’ll find it’s an expressive and powerful way to perform queries against Neo4j. Domain-Specific Steps Graph traversal is nice, but businesses and organizations tend to converse in domain-specific languages. For example, we wouldn’t normally ask “What is the vertex with the incoming edge of grape_type sharing the outgoing edge of this wine’s vertex?” but rather “What varietal is this wine?” Gremlin is already a language specific to the domain of querying graph databases, but what about making the language even more specific? Gremlin lets us do this by creating new steps that are semantically meaningful to the data stored in the graph.

g.V.filter{it.name=='Patty'}.friendsuggest.name​​ ​​==> Prancing Wolf Ice Wine 2007​​ ​​==> Prancing Wolf Kabinett 2002​​ Since varietal and friendsuggest are just normal Pipe-building steps, you can chain them together to make more interesting queries. The following finds the varietals that Patty’s friends like best: ​​g.V.filter{it.name=='Patty'}.friendsuggest.varietal.name​​ ​​==> riesling​​ Using Groovy metaprogramming to create new steps is a powerful force for crafting domain-specific languages. But like Gremlin itself, the practice can take some getting used to. Update, Delete, Done You’ve inserted and stepped through a graph, but what about updating and deleting data? It’s easy enough, once you find the vertex or edge you want to alter. Let’s add a weight to how much Alice likes the Prancing Wolf Ice Wine 2007. ​​gremlin> e=g.V.filter{it.name=='Alice'}.outE('likes').next()​​ ​​gremlin> e.weight = 95​​ ​​gremlin> e.save​​ We can remove the value just as easily. ​​


pages: 761 words: 80,914

Ansible: Up and Running: Automating Configuration Management and Deployment the Easy Way by Lorin Hochstein

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, cloud computing, continuous integration, Debian, DevOps, domain-specific language, don't repeat yourself, general-purpose programming language, Infrastructure as a Service, job automation, pull request, side project, smart transportation, web application

Declarative A type of programming language where the programmer describes the desired output, not the procedure for how to compute the output. Ansible’s playbooks are declarative. SQL is another example of a declarative language. Contrast with procedural languages, such as Java and Python. Deployment The process of bringing software up onto a live system. DevOps IT buzzword that gained popularity in the mid-2010s. Dry run See Check mode. DSL Domain-specific language. In systems that use DSLs, the user interacts with the systems by writing text files in the domain-specific language and then runs those files through the system. DSLs are not as powerful as general-purpose programming language, but (if designed well) they are easier to read and write than general-purpose programming language. Ansible exposes a DSL that uses YAML syntax. Dynamic inventory Source that provides Ansible with information about hosts and groups at playbook execution time.

When we talk about configuration management, we are typically talking about writing some kind of state description for our servers, and then using a tool to enforce that the servers are, indeed, in that state: the right packages are installed, configuration files contain the expected values and have the expected permissions, the right services are running, and so on. Like other configuration management tools, Ansible exposes a domain-specific language (DSL) that you use to describe the state of your servers. These tools also can be used for doing deployment as well. When people talk about deployment, they are usually referring to the process of taking software that was written in-house, generating binaries or static assets (if necessary), copying the required files to the server(s), and then starting up the services. Capistrano and Fabric are two examples of open-source deployment tools.


pages: 234 words: 63,522

Puppet Essentials by Felix Frank

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

cloud computing, Debian, DevOps, domain-specific language, Infrastructure as a Service, platform as a service, web application

You can visit https://docs.puppetlabs. com/guides/install_puppet/pre_install.html for more details. A platform-independent way to install Puppet is to get the puppet Ruby gem. This is fine for testing and managing single systems, but is not recommended for production use. After installing Puppet, you can use it to do something for you right away. Puppet is driven by manifests, the equivalent of scripts or programs, written in Puppet's domain-specific language (DSL). Let's start with the obligatory Hello world manifest: # hello_world.pp notify { 'Hello, world!': } Downloading the example code You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub. com/support and register to have the files e-mailed directly to you.

[ 206 ] Index A agents initializing, in cloud 185 resources, exporting to 141 anchor pattern about 90 URL 91 antipatterns avoiding 154, 155 apt-get command 8 arrays 15 autorequire feature 125 autoscaling feature about 198 certificates, managing 198-200 round trip times, limiting 200-202 autosigning URL 200 autosigning script 198 B backends selecting 165 URL, for online documentation 165 beaker about 105 URL 105 before metaparameter 19, 21, 24 C classes about 66 component classes, writing 73, 74 comprehensive classes, writing 71, 72 creating, with parameters 92 declaring 66, 67 defining 66, 67 definitions, nesting 82 differentiating, with defined types 69, 70 include keyword, preferring 93 parameterized classes, consequences 92, 93 class inheritance 149 cloud agents, initializing in 185 manifests, building for 187 cloud-provisioner module using 186 collectors used, for realizing resources 140, 141 component classes writing 73, 74 composite design 71 comprehensive classes writing 71, 72 configuration data structuring, in hierarchy 161, 162 containers events, passing between classes and defined types 83-85 limitations 86-89 limitations, mitigating 90 ordering 86 relationships, establishing among 83 containers, limitations anchor pattern 90 contain function 91 control structures adding, in manifest 13, 14 creates parameter 28 cron resource type 29 custom attribute 191 custom facts about 53 Facter, extending with 53-55 custom functions about 96 used, for refining custom module interface 126-128 custom module building 105 enhancing, through facts 125 implementing 106-109 interface, refining through custom functions 126-128 making, portable across platforms 128, 129 naming 106 using 106 utilities, creating for derived manifests 110 custom types 117 D data resources, converting to 172-174 data, defining in manifest consequences 159, 160 defined types about 66 creating 67-69 differentiating, with classes 69, 70 used, for exploiting array values 78-81 using 67-69 using, as macros 77, 78 using, as resource multiplexers 76 using, as resource wrappers 74, 75 dependency 20 documentation, modules 98, 99 domain-specific language (DSL) 8 dynamic configuration files templating 134 dynamic scoping 154 E enabled property 10 ensure property 10 environment.conf file 100 environment locations configuring 100, 101 environments maintaining 99, 100 modules, installing 101, 102 modules, obtaining 101, 102 used, for testing modules 104, 105 evaluation order circular dependencies, avoiding 21, 22 controlling 16 dependencies, declaring 17-20 error propagation 20 events about 23 passing, between classes and defined types 83-85 exec resource type 27 external facts using 55, 56 External Node Classifiers (ENCs) 174 F Faces 186 Facter example 62 extending, with custom facts 53-55 goals 57 systems, summarizing with 50, 51 facts URL, for documentation 125 used, for enhancing custom module 125 fact values accessing 52, 53 using 52, 53 flexibility, providing to classes about 148 class inheritance 149 inheriting class, naming 151 parameters, making safer through inheritance 151 [ 208 ] Forge modules' characteristics, identifying 130 URL 130 used, for searching modules 130 fqdn_rand function 41 fully qualified domain name (FQDN) 52 G group resource type 26 H hashes 14 Hiera arrays, handling 170-172 class parameter values, binding 167-169 configuring 163 data, storing 164 hashes, handling 170-172 lookups, defining 179 practical example 177, 178 using, in different contexts 175, 176 values, retrieving 165 values, using in manifest 165 working with simple values 166, 167 hiera_array function 170 hiera_hash function 171 hierarchy configuration data, structuring in 161, 162 I immutability, variables 14 include keyword preferring 93 Infrastructure as a Service (IaaS) 184 Infrastructure as Code paradigm 105 inheriting class naming 151 installation, modules 101, 102 instances method 123 M manifest about 182 control structures, adding in 13, 14 dry-testing 12 structure 9 manifest, and Hiera designs selecting between 175 manifest, building for cloud about 187 arbitrary configuration files, composing 194-196 certificate names, selecting 190, 191 distributed catalog, creating 191-194 functionality, mapping to nodes 187-189 instance deletions, handling 197, 198 metaparameters 18 model substantiating, with providers 59, 60 modules about 96 agent, enhancing through plugins 116, 117 best practices 102 content structure 97, 98 documentation 98, 99 generalization, avoiding 103 identifying, in Forge 130 important parts 96 installing 101, 102 manifest files, gathering 102, 103 obtaining 101, 102 searching, in Forge 130 testing 104 testing, with environments 104, 105 URL, for publishing 98 monolithic implementation 71 mount resource type 29, 30 N Nginx about 45 Phusion Passenger, using with 45, 46 nodes file 100 Notice keyword 20 [ 209 ] O operatingsystemrelease fact 53 output interpreting, of puppet apply command 11, 12 P Proudly sourced and uploaded by [StormRG] Kickass Torrents | TPB | ExtraTorrent | h33t parameterized classes consequences 92, 93 parameters versus properties 10 parser functions 96 performance bottlenecks avoiding, from templates 136 performance considerations about 42 basic tuning 46 Passenger, using with Nginx 45 switching, to Phusion Passenger 43, 44 Phusion Passenger switching to 43, 44 URL, for installation instructions 45 using, with Nginx 45, 46 Platform as a Service (PaaS) 184 plugins about 116 custom types, creating 118 custom types, naming 118 management commands, declaring 121 provider, adding 121 provider, allowing to prefetch existing resources 123, 124 provider functionality, implementing 122, 123 resource names, using 120 resource type interface, creating 119 sensible parameter hooks, designing 120 types, making robust 125 used, for enhancing modules agent 116, 117 plugins, types custom facts 116 parser functions 116 providers 116 types 116 processorcount fact 52 properties about 10 versus parameters 10 providerless resource types 61 provider parameter 10 providers model, substantiating with 59, 60 summarizing 61 Puppet about 182 installing 8 modules 96 typical scopes 182 URL 182 Puppet agent certificate, renewing 40 life cycle 38, 39 running, from cron 41 setting up 35-37 puppet apply command about 9, 31 output, interpreting of 11, 12 PuppetBoard 186 Puppet Dashboard 186 Puppet Explorer 186 Puppet Labs URL 8 URL, for advanced approaches 43 URL, for core resource types 61 URL, for style guide 52 URL, for system installation information 32 URL, for Troubleshooting section 47 puppetlabs-strings module URL 99 Puppet master about 31 configuration settings, inspecting 35 master machine, setting up 32 master manifest, creating 33, 34 tasks 32 puppetmaster system service 33 puppet module install command 101 Puppet support, for SSL CSR attributes URL 199 [ 210 ] Puppet, taking to cloud about 184 agents, initializing 185 cloud-provisioner module, using 186 Puppet toolchain 46 rspec-puppet module about 105 URL 105 R separate data storage need for 158 singletons 135 site manifest 33 SSL troubleshooting 47, 48 stdlib module 101 strings 15 subscribe metaparameter 23 successful provisioning, ensuring about 202 manifests, testing 204, 205 necessary relationships, adding 203 systems summarizing, with Facter 50, 51 S realize function 138, 139 redundancy saving, resource defaults used 152, 153 relationships, containers performance implications 89 require metaparameter 19 resource chaining 17 resource defaults used, for saving redundancy 152, 153 resource interaction implementing 22-24 resource parameters overriding 147, 148 resources about 10 converting, to data 172-174 exporting 142 exporting, to agents 141 importing 142 realizing, collectors used 140, 141 resources, exporting about 141 central firewall, maintaining 146 custom configuration, automating 144 hosts files, managing 144 master configuration, for storing exported resources 142 Nagios configuration, simplifying 145, 146 SSH host keys, exporting 143 resource type life cycle, agent side 58, 59 resource types cron 29 examining 25, 26 exec 27, 28 group 26 mount 29, 30 user 26 revocation 39 Roles and Profiles pattern 183 T templates performance bottlenecks, avoiding from 136 using 135, 136 template syntax learning 134, 135 transaction 57 Trusted Facts 189 types about 117 summarizing 61 type system 57 typical scopes, Puppet about 182 profiles 183, 184 roles 183, 184 U user resource type 26 utilities, custom module complexity, dealing 115, 116 configuration items, adding 111, 112 creating, for derived manifests 110 [ 211 ] customization, allowing 113 unwanted configuration items, removing 114, 115 W Warning keyword 20 V Y Vagrant 182 variables using 14 variable types about 14 arrays 15 hashes 14 strings 15 virtual resources creating 137, 138 yum command 8 [ 212 ] Thank you for buying Puppet Essentials About Packt Publishing Packt, pronounced 'packed', published its first book "Mastering phpMyAdmin for Effective MySQL Management" in April 2004 and subsequently continued to specialize in publishing highly focused books on specific technologies and solutions.


pages: 757 words: 193,541

The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services, Volume 2 by Thomas A. Limoncelli, Strata R. Chalup, Christina J. Hogan

active measures, Amazon Web Services, anti-pattern, barriers to entry, business process, cloud computing, commoditize, continuous integration, correlation coefficient, database schema, Debian, defense in depth, delayed gratification, DevOps, domain-specific language, en.wikipedia.org, fault tolerance, finite state, Firefox, Google Glasses, information asymmetry, Infrastructure as a Service, intermodal, Internet of things, job automation, job satisfaction, load shedding, loose coupling, Malcom McLean invented shipping containers, Marc Andreessen, place-making, platform as a service, premature optimization, recommendation engine, revision control, risk tolerance, side project, Silicon Valley, software as a service, sorting algorithm, statistical model, Steven Levy, supply-chain management, Toyota Production System, web application, Yogi Berra

., 345 Dickson model, 334 diff tool, 33 Differentiated services, 233 Direct measurements, 347–348 Direct orchestration, 213–214 DiRT (Disaster Recovery Testing), 316, 318, 320–323 Disaster preparedness, 307, 448–450 antifragile systems, 308–309 DiRT tests, 320–323 exercises, 330 fire drills, 312–313 implementation and logistics, 318–320 incident Command System, 323–329 mindset, 308–310 random testing, 314–315 risk reduction, 309–311 scope, 317–318 service launches, 158 service testing, 313–314 starting, 316–317 summary, 329–330 training for individuals, 311–312 training for organizations, 315–317 Disaster Recovery Testing (DiRT), 316, 318, 320–323 Disks access time, 26 caches, 106–107 failures, 132–133 Distributed computing and clouds cloud computing era, 469–472 conclusion, 472–473 dot-bomb era, 459–465 exercises, 473 first web era, 455–459 origins overview, 451–452 pre-web era, 452–455 second web era, 465–469 Distributed computing overview, 9–10 CAP Principle, 21–24 distributed state, 17–20 exercises, 30 load balancer with multiple backend replicas, 12–13 loosely coupled systems, 24–25 server trees, 16–17 servers with multiple backends, 14–15 simplicity importance, 11 speed issues, 26–29 summary, 29–30 visibility at scale, 10–11 Distributed denial-of-service (DDoS) attacks, 140 Distributed state, 17–20 Distributed version control systems (DVCSs), 265 Diurnal cycles, 332 Diurnal usage patterns, 359 Diversity, monitoring, 334 DNS deployment phase, 222 round robin, 72–73 Docker system, 61, 219 Documentation design documents. See Design documents design for operations, 43–44 service launches, 158 stakeholder interactions, 154 Doerr, John, 389 Domains, failure, 126–128 Domain-specific languages (DSLs), 244 DoS (denial-of-service) attacks, 140 Dot-bomb era (2000–2003), 459–465 Downsampling, monitoring, 339 Downtime containers, 61 pre-web era, 453 in upgrading live services, 225 Drain tool, 254 Draining process, 112 Drains, queue, 35–36 “DRAM Errors in the Wild: A Large-Scale Field Study” article, 134 DSLs (domain-specific languages), 244 Dual load balancers, 76 Durability in ACID term, 24 DVCSs (distributed version control systems), 265 Dynamic content with web servers, 70 Dynamic resource allocation, 138 Dynamic roll backs, 232 Dynamo system, 24 “Each Necessary, But Only Jointly Sufficient” article, 302 ECC (error-correcting code) memory, 131–132 Edge cases, 153 Edwards, Damon DevOps benefits, 172–173 DevOps Cafe podcast, 180, 188, 200 Effectiveness of caches, 105 80/20 rule for operational features, 47 Elements of Programming Style, 11 Eliminating tasks, 155 EMA (exponential moving average), 367, 379 Email alerts, 292–293 archives, 277 Embedded knowledge in DevOps, 177–178, 187 Emergency hotfixes, 240 Emergency issues, 160 Emergency Response (ER), 403, 426–428 Emergency tasks, 156 Employee human resources data updates example, 89–90 Empowering users, automation for, 253 Emptying queues, 35 Encryption in four-tier web service, 79 End-of-shift oncall responsibilities, 299 End-to-end process in service delivery, 200 Engagement defined, 366 measuring, 374–375 Enterprise Integration Practices: Designing, Building, and Deploying Messaging Solutions, 87 Environment-related files, 220 Ephemeral computing, 67 Ephemeral machines, 58 Erlang language, 236 Error Budgets, 152 case study, 396–399 DevOps, 184 Error-correcting code (ECC) memory, 131–132 Escalation alert messages, 345, 354–357 automated, 128–129 monitoring, 333 third-party, 298 Etsy blog, 256 EU Data Protection Directive platform selection factor, 63 requirements, 43 Eventual consistency, 21 Exception collection, 43–44 Exceptional situations.

However, if the people running the automation have no insight into how it works, it is much harder for them to figure out how and why it failed, and therefore handle the exception appropriately, than it would be if they were more involved in the process. This situation is common where the person handling the exception is not the person who wrote the automation. And at some point, that is the situation for all automation. * * * Terms to Know Domain-Specific Language (DSL): A language that was purpose-built for a particular use, such as system administration, mathematics, or text manipulation. Toil: Exhausting physical labor. * * * 12.1 Approaches to Automation There are three primary approaches to automation design. The first, and most common, approach is to automate using the “left-over” principle, where the automation handles as much as possible, with people being expected to handle whatever is left over.

Automation written in a compiled language typically scales better than the same automation written in a scripting language. Compiled languages often used by system administrators include C, C++, and Go. As described earlier, compiled languages are usually statically typed and catch more errors at compile time. 12.6.4 Configuration Management Languages Configuration management (CM) languages are domain-specific languages (DSLs) created specifically for system administration tasks. CM systems are created for maintaining the configuration of machines, from low-level settings such as the network configuration to high-level settings such as which services should run and their configuration files. Configuration languages are declarative. That is, the programmer writes code that describes how the world should be and the language figures out which changes are needed to achieve that goal.

Functional Programming in Scala by Paul Chiusano, Rúnar Bjarnason

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

domain-specific language, iterative process, loose coupling, sorting algorithm, type inference, web application

Functions of this type describe state actions that transform RNG states, and these state actions can be built up and combined using general-purpose functions. To make them convenient to talk about, let's make a type alias for the RNG state action data type: type Rand[+A] = RNG => (A, RNG) We can now turn methods such as RNG's nextInt into values of this type: val int: Rand[Int] = _.nextInt We want to start writing combinators that let us avoid explicitly passing along the RNG state. This will become a kind of domain-specific language that does all of this passing for us. For example, a simple RNG-transition is the unit action, which passes the RNG state through without using it, always returning a constant value rather than a random value. def unit[A](a: A): Rand[A] = rng => (a, rng) There is also map, for transforming the output of a state action without modifying the state itself. Remember, Rand[A] is just a type alias for a function type RNG => (A, RNG), so this is just a kind of function composition. def map[A,B](s: Rand[A])(f: A => B): Rand[B] = rng => { val (a, rng2) = s(rng) (f(a), rng2) } EXERCISE 5: Use map to generate an Int between 0 and n, inclusive: def positiveMax(n: Int): Rand[Int] www.it-ebooks.info 82 EXERCISE 6: Use map to reimplement RNG.double in a more elegant way.

The IO monad will be important for two reasons: It provides the most straightforward way of embedding imperative programming into FP, while preserving referential transparency and keeping pure code separate from what we'll call effectful code. We will be making an important distinction here in this chapter between effects and side effects. It illustrates a key technique for dealing with external effects—using pure functions to compute a description of an imperative computation, which is then executed by a separate interpreter. Essentially, we are crafting an embedded domain specific language (EDSL) for imperative programming. This is a powerful technique we'll be using throughout part 4; part of our goal is to equip you with the skills needed to begin crafting your own descriptions for interesting effectful programs you are faced with. 13.2 Factoring effects We are going to work our way up to the IO monad by first considering a very simple example, one we discussed in chapter 1 of this book: case class Player(name: String, score: Int) def printWinner(p: Player): Unit = println(p.name + " is the winner!")


pages: 141 words: 9,896

Pragmatic Guide to JavaScript by Christophe Porteneuve

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

barriers to entry, commoditize, domain-specific language, en.wikipedia.org, Firefox, web application, WebSocket

A Practical Guide to Successful Software Projects 2005 9780974514048 224 9781934356593 SQL Antipatterns: Avoiding the Pitfalls of Database Programming 2010 Test-Drive ASP.NET MVC 2010 9781934356531 296 The Agile Samurai: How Agile Masters Deliver Great Software 2010 9781934356586 280 Stripes ...and Java Web Development Is Fun Again TextMate: Power Editing for the Mac 2008 2007 9781934356555 300 9781934356210 9780978739232 352 375 208 The Definitive ANTLR Reference: Building Domain-Specific Languages 2007 9780978739256 384 The Passionate Programmer: Creating a Remarkable Career in Software Development 2009 9781934356340 200 ThoughtWorks Anthology Ubuntu Kung Fu: Tips, Tricks, Hints, and Hacks 2008 9781934356142 240 Web Design for Developers: A Programmer’s Guide to Design Tools and Techniques 2008 2009 9781934356135 300 Download from Wow! eBook <www.wowebook.com> 9781934356227 400 More Fun With Languages Seven Languages in Seven Weeks In this book you’ll get a hands-on tour of Clojure, Haskell, Io, Prolog, Scala, Erlang, and Ruby.


pages: 135 words: 31,098

ClojureScript: Up and Running by Stuart Sierra, Luke Vanderhart

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

domain-specific language, Firefox, MVC pattern, Paul Graham, web application

Because macros are evaluated during compilation, they can potentially convert an expression into a more-efficient form before it reaches the compiler. The ClojureScript compiler uses macros internally to produce more efficient code, but you are unlikely to encounter this situation in everyday programming. Summary Macros are an extremely powerful language tool, so powerful that they are rarely needed in everyday programming. However, for advanced tasks, such as defining new control structures or embedding domain-specific languages, they can be invaluable. This chapter has barely scratched the surface of what macros can do. For more examples, refer to books about Clojure. For even deeper exploration of macros, look to books on Common Lisp, such as Paul Graham’s classic On Lisp, available free online. Note that most other Lisps use the comma character instead of tilde for unquote. * * * [3] Technically, you can prevent evaluation of function arguments by wrapping each argument in an anonymous function, but this is syntactically cumbersome.


pages: 999 words: 194,942

Clojure Programming by Chas Emerick, Brian Carper, Christophe Grand

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, Benoit Mandelbrot, cloud computing, continuous integration, database schema, domain-specific language, don't repeat yourself, en.wikipedia.org, failed state, finite state, Firefox, game design, general-purpose programming language, Guido van Rossum, Larry Wall, mandelbrot fractal, Paul Graham, platform as a service, premature optimization, random walk, Ruby on Rails, Schrödinger's Cat, semantic web, software as a service, sorting algorithm, Turing complete, type inference, web application

In both cases, the answer is that Clojure did not import its syntactic foundations from other Lisp implementations on a whim; their adoption carries powerful benefits that are worth a minor shift in perspective: Prefixed operations used uniformly simplify the language’s syntax significantly and eliminate potential ambiguity from nontrivial expressions. The use of parentheses (as a textual representation of lists) is an outgrowth of Clojure being a homoiconic language. We’ll see what this means in Homoiconicity, but the ramifications of it are manifold: homoiconicity enables the development and use of metaprogramming and domain-specific language constructs simply unavailable in any programming language that is not homoiconic. After getting through an initial period of unfamiliarity, you will very likely find that Clojure’s syntax reduces the cognitive load necessary to read and write code. Quick: is << (bit-shift left) in Java executed before or after & (bitwise and) in order of operations? Every time a programmer has to pause and think about this (or look it up in a manual), every time a programmer has to go back and add grouping parentheses “just in case,” a mental page fault has occurred.

For example, Figure 1-1 shows some examples of textual language and possible transformations to their corresponding syntax trees.[7] Figure 1-1. Sample transformations from textual language to formal models These transformations from a textual manifestation of language to an AST are at the heart of how languages are defined, how expressive they are, and how well-suited they are to the purpose of relating to the world within which they are designed to be used. Much of the appeal of domain-specific languages springs from exactly this point: if you have a language that is purpose-built for a given field of use, those that have expertise in that field will find it far easier to define and express what they wish in that language compared to a general-purpose language. The downside of this approach is that most languages do not provide any way to control their ASTs; the correspondence between their textual syntax and their ASTs is defined solely by the language implementers.

See with-open, finally’s Lament for a discussion of with-open, the most commonly used with-* function in Clojure. [369] Perlis is relevant here, see Abstractions over Implementations [370] See Lazy seqs. [371] Other JDBC drivers will throw an exception when you attempt to read data from a closed connection; the SQLite driver appears to prefetch the first row for us, thus the single-row result. Korma Korma (http://sqlkorma.com) is an up-and-coming domain-specific language for working with relational databases in Clojure. It aims to provide a “batteries included” and Clojure-native database interaction experience; to that end, it takes care of generating SQL for many different popular databases, and handles administrative tasks like managing connection pooling via c3p0. For those familiar with Ruby’s ActiveRecord or a similar object-relational mapper, Korma should seem familiar, although it is decidedly not an Object-Relational Mapping framework.

HBase: The Definitive Guide by Lars George

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, bioinformatics, create, read, update, delete, Debian, distributed revision control, domain-specific language, en.wikipedia.org, fault tolerance, Firefox, Google Earth, place-making, revision control, smart grid, web application

Transactions make it possible to update multiple tables in an atomic fashion so that either all modifications are visible or none are visible. The RDBMS gives you the so-called ACID[14] properties, which means your data is strongly consistent (we will address this in greater detail in Consistency Models). Referential integrity takes care of enforcing relationships between various table schemas, and you get a domain-specific language, namely SQL, that lets you form complex queries over everything. Finally, you do not have to deal with how data is actually stored, but only with higher-level concepts such as table schemas, which define a fixed layout your application code can reference. This usually works very well and will serve its purpose for quite some time. If you are lucky, you may be the next hot topic on the Internet, with more and more users joining your site every day.

The code has access to the server local data and can be used to implement lightweight batch jobs, or use expressions to analyze or summarize data based on a variety of operators. Note Coprocessors were added to HBase in version 0.91.0. Finally, the system is integrated with the MapReduce framework by supplying wrappers that convert tables into input source and output targets for MapReduce jobs. Unlike in the RDBMS landscape, there is no domain-specific language, such as SQL, to query data. Access is not done declaratively, but purely imperatively through the client-side API. For HBase, this is mostly Java code, but there are many other choices to access the data from other programming languages. Implementation Bigtable [...] allows clients to reason about the locality properties of the data represented in the underlying storage. The data is stored in store files, called HFiles, which are persistent and ordered immutable maps from keys to values.

Using Cascading to insert data into HBase // read data from the default filesystem // emits two fields: "offset" and "line" Tap source = new Hfs(new TextLine(), inputFileLhs); // store data in an HBase cluster, accepts fields "num", "lower", and "upper" // will automatically scope incoming fields to their proper familyname, // "left" or "right" Fields keyFields = new Fields("num"); String[] familyNames = {"left", "right"}; Fields[] valueFields = new Fields[] {new Fields("lower"), new Fields("upper") }; Tap hBaseTap = new HBaseTap("multitable", new HBaseScheme(keyFields, familyNames, valueFields), SinkMode.REPLACE); // a simple pipe assembly to parse the input into fields // a real app would likely chain multiple Pipes together for more complex // processing Pipe parsePipe = new Each("insert", new Fields("line"), new RegexSplitter(new Fields("num", "lower", "upper"), " ")); // "plan" a cluster executable Flow // this connects the source Tap and hBaseTap (the sink Tap) to the parsePipe Flow parseFlow = new FlowConnector(properties).connect(source, hBaseTap, parsePipe); // start the flow, and block until complete parseFlow.complete(); // open an iterator on the HBase table we stuffed data into TupleEntryIterator iterator = parseFlow.openSink(); while(iterator.hasNext()) { // print out each tuple from HBase System.out.println( "iterator.next() = " + iterator.next() ); } iterator.close(); Cascading to Hive and Pig offers a Java API, as opposed to the domain-specific languages (DSLs) provided by the others. There are add-on projects that provide DSLs on top of Cascading. * * * [76] http://hive.apache.org/ [77] See the Hive wiki for more details on storage handlers. [78] The Hive wiki has a full explanation of the HBase integration into Hive. [79] http://pig.apache.org/ [80] Internally it uses the RowFilter class; see RowFilter. [81] The full details can be found on the Pig setup page.


pages: 834 words: 180,700

The Architecture of Open Source Applications by Amy Brown, Greg Wilson

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

8-hour work day, anti-pattern, bioinformatics, c2.com, cloud computing, collaborative editing, combinatorial explosion, computer vision, continuous integration, create, read, update, delete, David Heinemeier Hansson, Debian, domain-specific language, Donald Knuth, en.wikipedia.org, fault tolerance, finite state, Firefox, friendly fire, Guido van Rossum, linked data, load shedding, locality of reference, loose coupling, Mars Rover, MVC pattern, peer-to-peer, Perl 6, premature optimization, recommendation engine, revision control, Ruby on Rails, side project, Skype, slashdot, social web, speech recognition, the scientific method, The Wisdom of Crowds, web application, WebSocket

Nonetheless, each CI system has a new and different way of specifying the build and test commands to be run. In fact, this seems to be one of the reasons why so many basically identical CI systems exist: each language and community implements their own configuration system, tailored to their own build and test systems, and then layers on the same set of features above that system. Therefore, building a domain-specific language (DSL) capable of representing the options used by the few dozen commonly used build and test tool chains would go a long way toward simplifying the CI landscape. Common formats for build and test reporting: There is little agreement on exactly what information, in what format, a build and test system needs to provide. If a common format or standard could be developed it would make it much easier for continuous integration systems to offer both detailed and summary views across builds.

This brings up another challenge: each shared component needs to be able to reason about target specific properties in a generic way. For example, a shared register allocator needs to know the register file of each target and the constraints that exist between instructions and their register operands. LLVM's solution to this is for each target to provide a target description in a declarative domain-specific language (a set of .td files) processed by the tblgen tool. The (simplified) build process for the x86 target is shown in Figure 11.5. Figure 11.5: Simplified x86 Target Definition The different subsystems supported by the .td files allow target authors to build up the different pieces of their target. For example, the x86 back end defines a register class that holds all of its 32-bit registers named "GR32" (in the .td files, target specific definitions are all caps) like this: def GR32 : RegisterClass<[i32], 32, [EAX, ECX, EDX, ESI, EDI, EBX, EBP, ESP, R8D, R9D, R10D, R11D, R14D, R15D, R12D, R13D]> { … } This definition says that registers in this class can hold 32-bit integer values ("i32"), prefer to be 32-bit aligned, have the specified 16 registers (which are defined elsewhere in the .td files) and have some more information to specify preferred allocation order and other things.

The syntax was obscure, but in many ways resembled the sed and awk commands, at least conceptually. The m4 format was declarative: although it was possible to drop into the low-level raw language, in practice these details were hidden from the user. It isn't clear that this decision was correct or incorrect. I felt at the time (and still feel) that with complex systems it can be useful to implement what amounts to a Domain Specific Language (DSL) for building certain portions of that system. However, exposing that DSL to end users as a configuration methodology essentially converts all attempts to configure a system into a programming problem. Great power results from this, but at a non-trivial cost. 17.5. Other Considerations Several other architectural and development points deserve to be mentioned. 17.5.1. A Word About Optimizing Internet Scale Systems In most network-based systems there is a tension between the client and the server.


pages: 648 words: 108,814

Solr 1.4 Enterprise Search Server by David Smiley, Eric Pugh

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, bioinformatics, cloud computing, continuous integration, database schema, domain-specific language, en.wikipedia.org, fault tolerance, Firefox, information retrieval, Internet Archive, Ruby on Rails, web application, Y Combinator

solr-ruby versus rsolr For a lower-level client interface to Solr from Ruby environments, there are two libraries duking it out to be the client of choice. In one corner you have solr-ruby, which is the client library officially supported by the Apache Solr project. solr-ruby is fairly widely used, including providing the API to Solr used by the acts_as_solr Rails plugin we looked at previously. The new kid on the block is rsolr, which is a re-imagining of what a proper DSL (Domain Specific Language) would look like for interacting with Solr. rsolr is used by Blacklight OPAC as its interface to Solr. Both of these solutions are solid. However, rsolr is currently gaining more attention, has better documentation, and nice features such as a direct Embedded Solr connection through JRuby. rsolr also has support for using either curb (Ruby bindings to curl, a very fast HTTP library) or the standard Net::HTTP library for the HTTP transport layer.

See Dismax Solr request EdgeNGram analyzer 61 handler EdgeNGramFilterFactory 61 dismax query handler 131 EdgeNGramTokenizerFactory 61 dismax request handler 128 Elasticfox 276 Dismax Solr request handler Embedded-Solr 65 about 128 embedded Solr automatic phrase boosting 132, 133 legacy Lucene, upgrading from 237 boost functions, boosting 137, 138 using for rich clients 237 boost queries, boosting 134-137 using in in-process streaming 236, 237 debugQuery option used 129 EmbeddedSolrServer class 224 default search 140, 141 encoder attribute 59 DisjunctionMaxQuery 130 EnglishPorterFilter Factory, stemming 54 features, over standard handler 129 Entity tags 279 limited query syntax 131 ETag 279 min-should-match 138 ETL 78 mm query parameter 138 eval() function 238 phrase slop, configuring 134 existence (and non-existence) queries 107 distanceMeasure, spellchecker option 174 explicit mapping 56 distributed search 32 Extract Transform and Load. See ETL div(x,y), mathematical primitives 121 extraParams entry 242 doc element 93 docText field data 233 F document deleting 70 facet 146 documentCache 281 facet.date 151, 286 Domain Specific Language. See DSL examples 151 double element 92 facet.date.end 151 DoubleMetaphone, phonetic encoding facet.date.gap 151 algorithms 58 facet.date.hardend 151 DoubleMetaphoneFilterFactory analysis facet.date.other 152 filter, options facet.date.start 151 inject 59 facet.field 147 maxCodeLength 59 facet.limit 147 Drupal, options facet.method 148 Apache Solr Search integration module 251 facet.mincount 147 Solr, hosted by Acquia 252 facet.missing 148 DSL 269 facet.missing parameter 143 dynamic fields facet.offset 147 * fallback 46 facet.prefix 148, 156 about 45 facet.query 286 facet.query parameter 152, 153 E facet.sort 147 facet_counts 143 echoHandler, diagnostic parameter 98 faceted navigation 7, 141, 145, 153 echoParams 152 faceted search 149, 220, 221 echoParams, diagnostic parameter 98 [ 304 ] Download at Boykma.Com This material is copyright and is licensed for the sole use by William Anderson on 26th August 2009 4310 E Conway Dr.


pages: 593 words: 118,995

Relevant Search: With Examples Using Elasticsearch and Solr by Doug Turnbull, John Berryman

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

commoditize, crowdsourcing, domain-specific language, finite state, fudge factor, full text search, information retrieval, natural language processing, premature optimization, recommendation engine, sentiment analysis

In the future, when you want to search or interact with the tmdb index, you’ll reference tmdb/movie/ or tmdb/ in the path of the Elasticsearch URL. 3.4.1. Your first searches of the TMDB Elasticsearch index Now you can search! For this movie application, you need to figure out how to respond to user searches from your application’s search bar. To do this, you’ll use Elasticsearch’s Query domain-specific language (DSL), or Query DSL. The Query DSL tells Elasticsearch how to execute a search using a JSON format. Here you specify factors such as required clauses, clauses that shouldn’t be included, boosts, field weights, scoring functions, and other factors that control matching and ranking. The Query DSL can be thought of as the search engine’s SQL, a query language focused on ranked retrieval of flat, denormalized documents.

simple signals, 2nd Solr additive, with Boolean queries boosting feature mappings multiplicative, with function queries user ratings vs. filtering breadcrumb navigation browse experience browse interface, Yowl buckets section building signals bulk index API bulkMovies string business and domain awareness business concerns group business weight business-ranking logic BusinessScore C cast.name field, 2nd, 3rd, 4th, 5th cast.name scores cast.name.bigrammed field, 2nd, 3rd, 4th character filtering, 2nd, 3rd character offsets classic similarity classification features cleaning click-through rate co-occurrence counting cold-start problem COLLAB_FILTER filter, 2nd collaboration filtering, using co-occurrence counting search relevance and collation collocation extraction combining fields committed documents common words, removing completion field, 2nd completion suggester completion_analyzer completion_prefix variable complexphrase query parser compound queries, 2nd, 3rd concept search basic methods for building augmenting content with synonyms concept signals building using machine learning personalized search and configurations conflate tokens constant_score query, 2nd content augmentation curation engineer/curator pairing risk of miscommunication with content curator role of content curator exploring extracting into documents providing to search engine searching content group content weight, 2nd ContentScore control analysis controlling field matching converge conversion rate coord (coordinating factor), 2nd, 3rd, 4th, 5th, 6th, 7th copyField, 2nd, 3rd copy_to option, 2nd cosine similarity cross_fields, 2nd, 3rd searching, 2nd, 3rd, 4th Solr solving signal discordance with cuisine field cuisine_hifi field, 2nd cuisine_lofi field curation, search relevance and custom all field custom score query D data-driven culture debugging example search application Elasticsearch first searches with The Movie Database Python matching query matching analysis to solve matching issues comparing query to inverted index fixing by changing analyzers query parsing underlying strategy ranking computing weight explain feature scoring matches to measure relevance search term importance similarity vector-space model decay functions, 2nd deep paging default analyzer defType parameter delimiters acronyms modeling specificity phone numbers synonyms tokenizing geographic data tokenizing integers tokenizing melodies deployment, relevance-focused search application description field, 2nd, 3rd, 4th, 5th descriptive query directors field directors.name field, 2nd, 3rd, 4th directors.name score directors.name.bigrammed, 2nd, 3rd disable_coord option disabling tokenization discriminating fields DisjunctionMaximumQuery dismax, 2nd doc frequency, 2nd doc values document search and retrieval aggregations Boolean search facets filtering Lucene-based search positional and phrase matching ranked results relevance sorting document-ranking system documents analysis enhancement enrichment extraction flattening nested grouping similar matching meaning of scored search completion from documents being searched tokens as features of matching process meaning of documents dot character dot product, 2nd down-boosting title DSL (domain-specific language) E e-commerce search, 2nd easy_install utility edismax query parser, 2nd Elasticsearch example search application overview end sentinels engaged field engaged restaurants English analyzer overview reindexing with english_* filters english_bigrams analyzer english_keywords filter english_possessive_stemmer filter english_stemmer filter english_stop filter enrichment, 2nd ETL (extract, transform, load), 2nd every field gets a vote exact matching, 2nd, 3rd, 4th expert search, 2nd, 3rd explanation field external sources extract function, 2nd, 3rd extracting features extraction F faceted browsing overview Solr facet.prefix option facets, 2nd, 3rd fail fast, 2nd, 3rd, 4th fast vector highlighter feature modeling, 2nd feature selection feature space features creation of overview, 2nd, 3rd feedback at search box search completion search suggestions search-as-you-type business and domain awareness content curation risk of miscommunication with content curator role of content curator in search results listing grouping similar documents information presented snippet highlighting when there are no results search relevance and Solr faceted browsing field collapsing match phrase prefix relevance feedback feature mappings suggestion and highlighting components while browsing alternative results ordering breadcrumb navigation faceted browsing field boosts field collapsing overview Solr field discordance field mappings field normalization field scores, 2nd field synchronicity, signal modeling and field-by-field dismax field-centric methods, 2nd field-centric search, combining term-centric search and combining greedy search and conservative amplifiers like fields precision vs. recall Solr fieldNorms, 2nd, 3rd fields fieldType field_value_factor function fieldWeight, 2nd, 3rd, 4th filter clause filter element filter queries filtering Amazon-style collaborative overview using co-occurrence counting score shaping vs. boosting finite state transducer fire token first_name field floating-point numbers fragment_size parameter fudge factors full bulk command full search string full-text search full_name field function decay function queries, multiplicative boosting with Boolean queries vs.


pages: 485 words: 74,211

Developing Web Applications with Haskell and Yesod by Michael Snoyman

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

create, read, update, delete, database schema, Debian, domain-specific language, don't repeat yourself, full text search, MVC pattern, web application

Even so, there are significant differences. Instead of using regular expressions, Yesod matches on pieces of a route. Instead of having a one-way route-to-handler mapping, Yesod has an intermediate data type (called the route data type, or a type-safe URL) and creates two-way conversion functions. Coding this more advanced system manually is tedious and error prone. Therefore, Yesod defines a Domain Specific Language (DSL) for specifying routes, and provides Template Haskell functions to convert this DSL to Haskell code. This chapter will explain the syntax of the routing declarations, give you a glimpse of what code is generated for you, and explain the interaction between routing and handler functions. Route Syntax Instead of trying to shoe-horn route declarations into an existing syntax, Yesod’s approach is to use a simplified syntax designed just for routes.


pages: 1,201 words: 233,519

Coders at Work by Peter Seibel

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Ada Lovelace, bioinformatics, cloud computing, Conway's Game of Life, domain-specific language, don't repeat yourself, Donald Knuth, fault tolerance, Fermat's Last Theorem, Firefox, George Gilder, glass ceiling, Guido van Rossum, HyperCard, information retrieval, Larry Wall, loose coupling, Marc Andreessen, Menlo Park, Metcalfe's law, Perl 6, premature optimization, publish or perish, random walk, revision control, Richard Stallman, rolodex, Ruby on Rails, Saturday Night Live, side project, slashdot, speech recognition, the scientific method, Therac-25, Turing complete, Turing machine, Turing test, type inference, Valgrind, web application

Algorithms that depend on destructively upgrading state—they don't work well. Seibel: So if you were writing a big image processing work-flow system, then would you write the actual image transformations in some other language? Armstrong: I'd write them in C or assembler or something. Or I might actually write them in a dialect of Erlang and then cross-compile the Erlang to C. Make a dialect—this kind of domain-specific language kind of idea. Or I might write Erlang programs which generate C programs rather than writing the C programs by hand. But the target language would be C or assembler or something. Whether I wrote them by hand or generated them would be the interesting question. I'm tending toward automatically generating C rather than writing it by hand because it's just easier. But I'd use an Erlang structure.

Seibel: So the right path, in your view, is getting back to that kind of work in automatic parallelization? Allen: Right, but it has to be done against a higher-level language than we have now. Seibel: And the wrong path is finding better ways for people to express parallelism explicitly? Allen: Well, I think we would eventually realize that we've created more of a mess than we had. But we do need higher-level languages and there certainly are domain-specific languages, and ways of developing things, which are really quite wonderful. But we have to be willing to try and take advantage of that, but also take advantage of the integration of systems and the fact that data's coming from everywhere. It's no longer encapsulated with the program, the code. We're seeing now, I think, vast amounts of data, which is accessible. And it's numeric data as well as the informational kinds of data, and will be stored all over the globe, especially if you're working in some of the bioinformatics kind of stuff.


pages: 509 words: 92,141

The Pragmatic Programmer by Andrew Hunt, Dave Thomas

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

A Pattern Language, Broken windows theory, business process, buy low sell high, c2.com, combinatorial explosion, continuous integration, database schema, domain-specific language, don't repeat yourself, Donald Knuth, general-purpose programming language, George Santayana, Grace Hopper, if you see hoof prints, think horses—not zebras, index card, loose coupling, Menlo Park, MVC pattern, premature optimization, Ralph Waldo Emerson, revision control, Schrödinger's Cat, slashdot, sorting algorithm, speech recognition, traveling salesman, urban decay, Y2K

Given that most applications exceed their expected lifetimes, you're probably better off biting the bullet and adopting the more complex and readable language up front. The initial effort will be repaid many times in reduced support and maintenance costs. Related sections include: Metaprogramming, page 144 Challenges Could some of the requirements of your current project be expressed in a domain-specific language? Would it be possible to write a compiler or translator that could generate most of the code required? If you decide to adopt mini-languages as a way of programming closer to the problem domain, you're accepting that some effort will be required to implement them. Can you see ways in which the framework you develop for one project can be reused in others? Exercises 5. We want to implement a mini-language to control a simple drawing package (perhaps a turtle-graphics system).

Programming Android by Zigurd Mednieks, Laird Dornin, G. Blake Meike, Masumi Nakamura

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

anti-pattern, business process, conceptual framework, create, read, update, delete, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, Google Earth, interchangeable parts, iterative process, loose coupling, MVC pattern, revision control, RFID, web application

While this is certainly among the most exciting technologies in Android, it is definitely at the edge of the map. It also appears that, just before the final beta release, the interface underwent major changes. Much of the code and many of the suggestions found on the Web are obsolete and no longer work. The API V1_r2 release is an implementation of OpenGL ES 1.0 and much of ES 1.1. It is essentially a domain-specific language embedded in Java. Someone who has been doing gaming UIs for a while is likely to be much more comfortable developing Android OpenGL programs than a Java programmer, even a Java UI expert. Before discussing the OpenGL graphics library itself, we should take a minute to consider exactly how pixels drawn with OpenGL appear on the display. Thus far, this chapter has discussed the intricate View framework that Android uses to organize and represent objects on the screen.


pages: 458 words: 135,206

CTOs at Work by Scott Donaldson, Stanley Siegel, Gary Donaldson

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, bioinformatics, business intelligence, business process, call centre, centre right, cloud computing, computer vision, connected car, crowdsourcing, data acquisition, distributed generation, domain-specific language, glass ceiling, pattern recognition, Pluto: dwarf planet, QR code, Richard Feynman, Richard Feynman, Ruby on Rails, shareholder value, Silicon Valley, Skype, smart grid, smart meter, software patent, thinkpad, web application, zero day, zero-sum game

I'll look at a problem and I'll think about it for a second, and I'll say, “There is a problem in a completely different domain, and if you apply the principles from that problem to this one, this is what you'll get.” An example is people will say, “A 10X improvement in IT management? That's not possible.” And my response is, “Well, relational databases did it for data. Why can't we do it for IT management? How did they do it? They did it in domain-specific languages for data management.” And the third gift I have is for understanding a business problem, converting it into technology, and then explaining back to business people in business terms. I don't know where that came from. And I'm just very grateful for those gifts. Whenever I see something new, my first reaction is, “How do I make money with this? How I do make it a good idea?” It's never, “Is this a good idea?”


pages: 655 words: 141,257

Programming Android: Java Programming for the New Generation of Mobile Devices by Zigurd Mednieks, Laird Dornin, G. Blake Meike, Masumi Nakamura

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

anti-pattern, business process, conceptual framework, create, read, update, delete, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, Google Earth, interchangeable parts, iterative process, loose coupling, MVC pattern, revision control, RFID, web application, yellow journalism

While this is certainly among the most exciting technologies in Android, it is definitely at the edge of the map. The good news is that, as of the Honeycomb release, OpenGL has been fully integrated into Android graphics. While early versions of Android supported OpenGL 1.0 and 1.1, according to the documentation, Honeycomb not only supports OpenGL 2.0, but also uses it as the basis for rendering View objects. OpenGL is essentially a domain-specific language embedded in Java. Someone who has been doing gaming UIs for a while is likely to be much more comfortable developing Android OpenGL programs than a Java programmer, even a Java UI expert. Before discussing the OpenGL graphics library itself, we should take a minute to consider exactly how pixels drawn with OpenGL appear on the display. Thus far, this chapter has discussed the intricate View framework that Android uses to organize and represent objects on the screen.


pages: 1,025 words: 150,187

ZeroMQ by Pieter Hintjens

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

anti-pattern, carbon footprint, cloud computing, Debian, distributed revision control, domain-specific language, factory automation, fault tolerance, fear of failure, finite state, Internet of things, iterative process, premature optimization, profit motive, pull request, revision control, RFC: Request For Comment, Richard Stallman, Skype, smart transportation, software patent, Steve Jobs, Valgrind, WebSocket

We’ll see more extensive examples later, because I really do believe that code generation is crucial knowledge for large-scale work. In Chapter 4, we developed the Majordomo Protocol (MDP) and wrote clients, brokers, and workers for that. Now, could we generate those pieces mechanically, by building our own interface description language and code generators? When we write a GSL model, we can use any semantics we like. In other words, we can invent domain-specific languages on the spot. I’ll invent a couple—see if you can guess what they represent: slideshow name = Cookery level 3 page title = French Cuisine item = Overview item = The historical cuisine item = The nouvelle cuisine item = Why the French live longer page title = Overview item = Soups and salads item = Le plat principal item = Béchamel and other sauces item = Pastries, cakes, and quiches item = Soufflé - cheese to strawberry Now, how about this one?

Scala in Action by Nilanjan Raychaudhuri

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

continuous integration, create, read, update, delete, database schema, domain-specific language, don't repeat yourself, en.wikipedia.org, failed state, fault tolerance, general-purpose programming language, index card, MVC pattern, type inference, web application

The only contract is to implement an iterator method: class FileAsIterable { def iterator = scala.io.Source.fromFile("someFile.txt").getLines() } Now if you mix in the Scala Iterable, your new FileAsIterable will become a Scala Iterable and will start supporting all the Iterable methods: val newIterator = new FileAsIterable with Iterable[String] newIterator.foreach { line => println(line) } In this case you’re using the foreach method defined in the Iterable trait and printing each line in the file. Scala version 2.10 adds support for a Dynamic[10] type. Using this feature you can dynamically add methods and fields to a type at runtime. This is very similar to the method_missing feature of Ruby and is quite useful if you’re building a domain-specific language (DSL). For example, Scala map is a collection of key value pairs and if you want to access the value associated with a key you can do something like the following: 10 “SIP-17 Type Dynamic,” http://docs.scala-lang.org/sips/pending/type-dynamic.html. val someMap = Map("foo" -> 1, "bar" -> 2) someMap.get("foo") Here someMap is a collection of two key value pairs and someMap.get("foo") will return 1.


pages: 509 words: 132,327

Rise of the Machines: A Cybernetic History by Thomas Rid

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

1960s counterculture, A Declaration of the Independence of Cyberspace, agricultural Revolution, Albert Einstein, Alistair Cooke, Apple II, Apple's 1984 Super Bowl advert, back-to-the-land, Berlin Wall, British Empire, Brownian motion, Buckminster Fuller, business intelligence, Claude Shannon: information theory, conceptual framework, connected car, domain-specific language, Douglas Engelbart, Douglas Engelbart, dumpster diving, Extropian, full employment, game design, global village, Haight Ashbury, Howard Rheingold, Jaron Lanier, job automation, John Markoff, John von Neumann, Kevin Kelly, Marshall McLuhan, Menlo Park, Mother of all demos, new economy, New Journalism, Norbert Wiener, offshore financial centre, oil shale / tar sands, pattern recognition, RAND corporation, Silicon Valley, Simon Singh, speech recognition, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, technoutopianism, Telecommunications Act of 1996, telepresence, V2 rocket, Vernor Vinge, Whole Earth Catalog, Whole Earth Review, Y2K, Yom Kippur War, Zimmermann PGP

He also was a former member of the Homebrew Computer Club. Hughes was just under thirty, with blond hair halfway down his back and a long wispy beard. He had studied math at Berkeley. In May 1992, Hughes came down to Santa Cruz from Berkeley to hunt for a house. But the two men were drawn into their common passion: crypto anarchy. “We spent three intense days talking about math, protocols, domain specific languages, secure anonymous systems,” May said later. “Man, it was fun.” The crypto rebels had been inspired by Martin Gardner’s famous 1977 column from Scientific American, fifteen years after it came out. “Wow, this is really mind-blowing,”27 May had thought when he read the piece. May and Hughes began to rope in others. A group of sixteen people started meeting every Saturday in an office building near Palo Alto full of small tech start-ups.

Exploring ES6 - Upgrade to the next version of JavaScript by Axel Rauschmayer

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

anti-pattern, domain-specific language, en.wikipedia.org, Firefox, Google Chrome, MVC pattern, web application, WebSocket

Template strings are known statically (at compile time), substitutions are only known at runtime. The tag function can do with its parameters as it pleases: It can completely ignore the template strings, return values of any type, etc. 9.3 Examples of using tagged templates To understand what tagged templates are good for, let’s look at examples. You’ll find that tagged templates allow you to implement custom embedded sub-languages (which are sometimes called domain-specific languages) with little effort, because JavaScript does much of the parsing for you. You only have to write a function that receives the results. Some of the following examples are borrowed from the original proposal¹ for template literals and tagged templates, which refers to them via their old name, quasi-literals. 9.3.1 Raw strings ES6 includes the tag function String.raw for raw strings, where backslashes have no special meaning: let str = String.raw`This is a text with multiple lines.


pages: 461 words: 125,845

This Machine Kills Secrets: Julian Assange, the Cypherpunks, and Their Fight to Empower Whistleblowers by Andy Greenberg

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Apple II, Ayatollah Khomeini, Berlin Wall, Bill Gates: Altair 8800, Burning Man, Chelsea Manning, computerized markets, crowdsourcing, cryptocurrency, domain-specific language, drone strike, en.wikipedia.org, fault tolerance, hive mind, Jacob Appelbaum, Julian Assange, Mahatma Gandhi, Mohammed Bouazizi, nuclear winter, offshore financial centre, pattern recognition, profit motive, Ralph Nader, Richard Stallman, Robert Hanssen: Double agent, Silicon Valley, Silicon Valley ideology, Skype, social graph, statistical model, stem cell, Steve Jobs, Steve Wozniak, Steven Levy, Vernor Vinge, We are Anonymous. We are Legion, We are the 99%, WikiLeaks, X Prize, Zimmermann PGP

So when May suggested Hughes use his home as a base for a few days while looking for a place to live, the pair hit it off like Marx and Engels. Hughes neglected his real estate search to wander with May through Santa Cruz’s redwood forests and beaches while the older ex-scientist unloaded several years’ worth of bottled-up ideas. “We spent three intense days talking about math, protocols, domain specific languages, secure anonymous systems,” says May. “Man, it was fun.” Just a week after Zimmermann released PGP 2.0, Hughes and May invited forty of their favorite coders and cryptographers to Hughes’s newly purchased home in Oakland. About twenty of them showed up, and the group, mostly wearing its de facto uniform of beards and ponytails, crowded into the furniture-free living room and sat cross-legged on his floor.


pages: 1,202 words: 144,667

The Linux kernel primer: a top-down approach for x86 and PowerPC architectures by Claudia Salzberg Rodriguez, Gordon Fischer, Steven Smolski

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Debian, domain-specific language, en.wikipedia.org, recommendation engine, Richard Stallman

You can add a directory to set obj-y, which indicates that it needs to descend into the specified directory: Obj-$(CONFIG_FOO) += /foo If /foo does not exist, make complaints with the following: Make[2]: *** No rule to make target '<dir>/foo/Makefile'. Stop. CML2 Where does the configuration program that you navigate when choosing kernel options get the information? The kbuild system depends on CML2, which is a domain-specific language designed for kernel configuration. CML2 creates a rulebase that an interpreter then reads and uses to generate the config file. This file covers the syntax and semantics of the language. The CML2 rulebase that is read by configuration programs is stored in files called defconfig and Kconfig. The defconfig files are found at the root of the architecture-specific directories, arch/*/. The Kconfig files are found in most other subdirectories.


pages: 933 words: 205,691

Hadoop: The Definitive Guide by Tom White

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Amazon Web Services, bioinformatics, business intelligence, combinatorial explosion, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, full text search, Grace Hopper, information retrieval, Internet Archive, linked data, loose coupling, openstreetmap, recommendation engine, RFID, SETI@home, social graph, web application

How a Flow translates to chained MapReduce jobs Because of this, developers can build applications of arbitrary granularity. They can start with a small application that just filters a logfile, but then can iteratively build up more features into the application as needed. Since Cascading is an API and not a syntax like strings of SQL, it is more flexible. First off, developers can create domain-specific languages (DSLs) using their favorite language, like Groovy, JRuby, Jython, Scala, and others (see the project site for examples). Second, developers can extend various parts of Cascading, like allowing custom Thrift or JSON objects to be read and written to and allowing them to be passed through the tuple stream. Hadoop and Cascading at ShareThis ShareThis is a sharing network that makes it simple to share any online content.


pages: 348 words: 39,850

Data Scientists at Work by Sebastian Gutierrez

Amazon: amazon.comamazon.co.ukamazon.deamazon.fr

Albert Einstein, algorithmic trading, Bayesian statistics, bioinformatics, bitcoin, business intelligence, chief data officer, clean water, cloud computing, commoditize, computer vision, continuous integration, correlation does not imply causation, creative destruction, crowdsourcing, data is the new oil, DevOps, domain-specific language, Donald Knuth, follow your passion, full text search, informal economy, information retrieval, Infrastructure as a Service, Intergovernmental Panel on Climate Change (IPCC), inventory management, iterative process, lifelogging, linked data, Mark Zuckerberg, microbiome, Moneyball by Michael Lewis explains big data, move fast and break things, move fast and break things, natural language processing, Network effects, nuclear winter, optical character recognition, pattern recognition, Paul Graham, personalized medicine, Peter Thiel, pre–internet, quantitative hedge fund, quantitative trading / quantitative finance, recommendation engine, Renaissance Technologies, Richard Feynman, Richard Feynman, self-driving car, side project, Silicon Valley, Skype, software as a service, speech recognition, statistical model, Steve Jobs, stochastic process, technology bubble, text mining, the scientific method, web application

Lenaghan: In a word, I’d tell them that what we do is ingest, ­transformation, and contextualization. We ingest enormous amounts of location data—on the order of 50 billion records a month. Then we essentially do large joins against our geospatial layer. We run many different types of classifiers on the geospatial layer to determine whether or not a particular tile is residential or retail or mixed-use. Then we apply a domain-specific language we’ve developed to build these audiences into profiles that not only contextualize the places but also add more insight into the patterns we see in the aggregate movement data. Gutierrez: How would you explain what you do in more qualitative terms to a five-year-old? Lenaghan: Five-year-olds are increasingly sophisticated about mobile devices these days. I’d say to a five-year-old that we are mappers of the world.


PostGIS in Action, 2nd Edition by Regina O. Obe, Leo S. Hsu

call centre, crowdsourcing, database schema, Debian, domain-specific language, en.wikipedia.org, Firefox, Google Earth, job automation, McMansion, megacity, Mercator projection, Network effects, openstreetmap, planetary scale, profit maximization, Ruby on Rails, Skype, South of Market, San Francisco, traveling salesman, web application

Imagine that you had to solve the shortest route or TSP problem on some set of data using just a conventional programming language. Without PostGIS or pgRouting, you’d have to define your own data structure, code the algorithm, and find a nice way to present the solution. Should the nature of your data change, you’d have to repeat the process. In the next sections, we’ll explore PL languages. The marriage of PL languages and SQL combines the expressiveness of an all-purpose or domain-specific language suited for certain classes of problems with the power of SQL. 16.2 Extending PostgreSQL with PLs One thing that makes PostgreSQL unique among the various relational databases is its pluggable procedural language (PL) architecture. Many Samaritans have created PL handlers for PostgreSQL, allowing you to write database stored functions in languages like Perl, Python, Java, TCL, R, JavaScript, and Sh (shell script), in addition to the built-in C, PL/pgSQL, and SQL.