簡單構建工具 · Scala 課堂! - 從 ? 到分布式服務

這堂課將概述SBT！具體議題包括： [TOC=2,2] ## 關于SBT SBT是一個現代化的構建工具。雖然它由Scala編寫并提供了很多Scala便利，但它是一個通用的構建工具。 ## 為什么選擇SBT？ * 明智的依賴管理 * 使用Ivy做依賴管理 * “只在請求時更新”的模型 * 對創建任務全面的Scala語言支持 * 連續執行命令 * 在項目上下文內啟動解釋器 ## 入門 **譯注**?最新的SBT安裝方式請參考?[scala-sbt的文檔](http://www.scala-sbt.org/release/docs/Getting-Started/Setup.html) * 下載jar包?[地址](http://code.google.com/p/simple-build-tool/downloads/list) * 創建一個調用這個jar的SBT shell腳本，例如 ~~~ java -Xmx512M -jar sbt-launch.jar "$@" ~~~ * 確保它是可執行的，并在你的path下 * 運行sbt來創建項目 ~~~ [local ~/projects]$ sbt Project does not exist, create new project? (y/N/s) y Name: sample Organization: com.twitter Version [1.0]: 1.0-SNAPSHOT Scala version [2.7.7]: 2.8.1 sbt version [0.7.4]: Getting Scala 2.7.7 ... :: retrieving :: org.scala-tools.sbt#boot-scala confs: [default] 2 artifacts copied, 0 already retrieved (9911kB/221ms) Getting org.scala-tools.sbt sbt_2.7.7 0.7.4 ... :: retrieving :: org.scala-tools.sbt#boot-app confs: [default] 15 artifacts copied, 0 already retrieved (4096kB/167ms) [success] Successfully initialized directory structure. Getting Scala 2.8.1 ... :: retrieving :: org.scala-tools.sbt#boot-scala confs: [default] 2 artifacts copied, 0 already retrieved (15118kB/386ms) [info] Building project sample 1.0-SNAPSHOT against Scala 2.8.1 [info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7 > ~~~ 可以看到它已經以較好的形式創建了項目的快照版本。 ## 項目布局 * 項目 – 項目定義文件 * project/build/.scala – 主項目定義文件 * project/build.properties – 項目、sbt和Scala版本定義 * src/main – 你的應用程序代碼出現在這里，在子目錄表明代碼的語言（如src/main/scala, src/main/java） * src/main/resources – 你想要添加到jar包中的靜態文件（如日志配置） * src/test – 就像src/main，不過是對測試 * lib_managed – 你的項目依賴的jar文件。由sbt update時填充 * target – 生成物的目標路徑（如自動生成的thrift代碼，類文件，jar包） ## 添加一些代碼我們將為簡單的tweet消息創建一個簡單的JSON解析器。將以下代碼加在這個文件中 src/main/scala/com/twitter/sample/SimpleParser.scala ~~~ package com.twitter.sample case class SimpleParsed(id: Long, text: String) class SimpleParser { val tweetRegex = "\"id\":(.*),\"text\":\"(.*)\"".r def parse(str: String) = { tweetRegex.findFirstMatchIn(str) match { case Some(m) => { val id = str.substring(m.start(1), m.end(1)).toInt val text = str.substring(m.start(2), m.end(2)) Some(SimpleParsed(id, text)) } case _ => None } } } ~~~ 這段代碼丑陋并有bug，但應該能夠編譯通過。 ## 在控制臺中的測試 SBT既可以用作命令行腳本，也可以作為構建控制臺。我們將主要利用它作為構建控制臺，不過大多數命令可以作為參數傳遞給SBT獨立運行，如 ~~~ sbt test ~~~ 需要注意如果一個命令需要參數，你需要使用引號包括住整個參數路徑，例如 ~~~ sbt 'test-only com.twitter.sample.SampleSpec' ~~~ 這種方式很奇怪。不管怎樣，要開始我們的代碼工作了，啟動SBT吧 ~~~ [local ~/projects/sbt-sample]$ sbt [info] Building project sample 1.0-SNAPSHOT against Scala 2.8.1 [info] using sbt.DefaultProject with sbt 0.7.4 and Scala 2.7.7 > ~~~ SBT允許你啟動一個Scala REPL并加載所有項目依賴。它會在啟動控制臺前編譯項目的源代碼，從而為我們提供一個快速測試解析器的工作臺。 ~~~ > console [info] [info] == compile == [info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed. [info] Compiling main sources... [info] Nothing to compile. [info] Post-analysis: 3 classes. [info] == compile == [info] [info] == copy-test-resources == [info] == copy-test-resources == [info] [info] == test-compile == [info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed. [info] Compiling test sources... [info] Nothing to compile. [info] Post-analysis: 0 classes. [info] == test-compile == [info] [info] == copy-resources == [info] == copy-resources == [info] [info] == console == [info] Starting scala interpreter... [info] Welcome to Scala version 2.8.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_22). Type in expressions to have them evaluated. Type :help for more information. scala> ~~~ 我們代碼編譯通過了，并提供了典型的Scala提示符。我們將創建一個新的解析器，一個tweet以確保其“能工作” ~~~ scala> import com.twitter.sample._ import com.twitter.sample._ scala> val tweet = """{"id":1,"text":"foo"}""" tweet: java.lang.String = {"id":1,"text":"foo"} scala> val parser = new SimpleParser parser: com.twitter.sample.SimpleParser = com.twitter.sample.SimpleParser@71060c3e scala> parser.parse(tweet) res0: Option[com.twitter.sample.SimpleParsed] = Some(SimpleParsed(1,"foo"})) scala> ~~~ ## 添加依賴我們簡單的解析器對這個非常小的輸入集工作正常，但我們需要添加更多的測試并讓它出錯。第一步是在我們的項目中添加specs測試庫和一個真正的JSON解析器。要做到這一點，我們必須超越默認的SBT項目布局來創建一個項目。 SBT認為project/build目錄中的Scala文件是項目定義。添加以下內容到這個文件中project/build/SampleProject.scala ~~~ import sbt._ class SampleProject(info: ProjectInfo) extends DefaultProject(info) { val jackson = "org.codehaus.jackson" % "jackson-core-asl" % "1.6.1" val specs = "org.scala-tools.testing" % "specs_2.8.0" % "1.6.5" % "test" } ~~~ 一個項目定義是一個SBT類。在上面例子中，我們擴展了SBT的DefaultProject。這里是通過val聲明依賴。SBT使用反射來掃描項目中的所有val依賴，并在構建時建立依賴關系樹。這里使用的語法可能是新的，但本質和Maven依賴是相同的 ~~~ <dependency> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-core-asl</artifactId> <version>1.6.1</version> </dependency> <dependency> <groupId>org.scala-tools.testing</groupId> <artifactId>specs_2.8.0</artifactId> <version>1.6.5</version> <scope>test</scope> </dependency> ~~~ 現在可以下載我們的項目依賴了。在命令行中（而不是sbt console中）運行sbt update ~~~ [local ~/projects/sbt-sample]$ sbt update [info] Building project sample 1.0-SNAPSHOT against Scala 2.8.1 [info] using SampleProject with sbt 0.7.4 and Scala 2.7.7 [info] [info] == update == [info] :: retrieving :: com.twitter#sample_2.8.1 [sync] [info] confs: [compile, runtime, test, provided, system, optional, sources, javadoc] [info] 1 artifacts copied, 0 already retrieved (2785kB/71ms) [info] == update == [success] Successful. [info] [info] Total time: 1 s, completed Nov 24, 2010 8:47:26 AM [info] [info] Total session time: 2 s, completed Nov 24, 2010 8:47:26 AM [success] Build completed successfully. ~~~ 你會看到sbt檢索到specs庫。現在還增加了一個lib_managed目錄，并且在lib_managed/scala_2.8.1/test目錄中包含 specs_2.8.0-1.6.5.jar ## 添加測試現在有了測試庫，可以把下面的測試代碼寫入src/test/scala/com/twitter/sample/SimpleParserSpec.scala文件 ~~~ package com.twitter.sample import org.specs._ object SimpleParserSpec extends Specification { "SimpleParser" should { val parser = new SimpleParser() "work with basic tweet" in { val tweet = """{"id":1,"text":"foo"}""" parser.parse(tweet) match { case Some(parsed) => { parsed.text must be_==("foo") parsed.id must be_==(1) } case _ => fail("didn't parse tweet") } } } } ~~~ 在SBT控制臺中運行test ~~~ > test [info] [info] == compile == [info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed. [info] Compiling main sources... [info] Nothing to compile. [info] Post-analysis: 3 classes. [info] == compile == [info] [info] == test-compile == [info] Source analysis: 0 new/modified, 0 indirectly invalidated, 0 removed. [info] Compiling test sources... [info] Nothing to compile. [info] Post-analysis: 10 classes. [info] == test-compile == [info] [info] == copy-test-resources == [info] == copy-test-resources == [info] [info] == copy-resources == [info] == copy-resources == [info] [info] == test-start == [info] == test-start == [info] [info] == com.twitter.sample.SimpleParserSpec == [info] SimpleParserSpec [info] SimpleParser should [info] + work with basic tweet [info] == com.twitter.sample.SimpleParserSpec == [info] [info] == test-complete == [info] == test-complete == [info] [info] == test-finish == [info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0 [info] [info] All tests PASSED. [info] == test-finish == [info] [info] == test-cleanup == [info] == test-cleanup == [info] [info] == test == [info] == test == [success] Successful. [info] [info] Total time: 0 s, completed Nov 24, 2010 8:54:45 AM > ~~~ 我們的測試通過了！現在，我們可以增加更多。運行觸發動作是SBT提供的優秀特性之一。在動作開始添加一個波浪線會啟動一個循環，在源文件發生變化時重新運行動作。讓我們運行 ~test 并看看會發生什么吧。 ~~~ [info] == test == [success] Successful. [info] [info] Total time: 0 s, completed Nov 24, 2010 8:55:50 AM 1\. Waiting for source changes... (press enter to interrupt) ~~~ 現在，讓我們添加下面的測試案例 ~~~ "reject a non-JSON tweet" in { val tweet = """"id":1,"text":"foo"""" parser.parse(tweet) match { case Some(parsed) => fail("didn't reject a non-JSON tweet") case e => e must be_==(None) } } "ignore nested content" in { val tweet = """{"id":1,"text":"foo","nested":{"id":2}}""" parser.parse(tweet) match { case Some(parsed) => { parsed.text must be_==("foo") parsed.id must be_==(1) } case _ => fail("didn't parse tweet") } } "fail on partial content" in { val tweet = """{"id":1}""" parser.parse(tweet) match { case Some(parsed) => fail("didn't reject a partial tweet") case e => e must be_==(None) } } ~~~ 在我們保存文件后，SBT會檢測到變化，運行測試，并通知我們的解析器有問題 ~~~ [info] == com.twitter.sample.SimpleParserSpec == [info] SimpleParserSpec [info] SimpleParser should [info] + work with basic tweet [info] x reject a non-JSON tweet [info] didn't reject a non-JSON tweet (Specification.scala:43) [info] x ignore nested content [info] 'foo","nested":{"id' is not equal to 'foo' (SimpleParserSpec.scala:31) [info] + fail on partial content ~~~ 因此，讓我們返工實現真正的JSON解析器 ~~~ package com.twitter.sample import org.codehaus.jackson._ import org.codehaus.jackson.JsonToken._ case class SimpleParsed(id: Long, text: String) class SimpleParser { val parserFactory = new JsonFactory() def parse(str: String) = { val parser = parserFactory.createJsonParser(str) if (parser.nextToken() == START_OBJECT) { var token = parser.nextToken() var textOpt:Option[String] = None var idOpt:Option[Long] = None while(token != null) { if (token == FIELD_NAME) { parser.getCurrentName() match { case "text" => { parser.nextToken() textOpt = Some(parser.getText()) } case "id" => { parser.nextToken() idOpt = Some(parser.getLongValue()) } case _ => // noop } } token = parser.nextToken() } if (textOpt.isDefined && idOpt.isDefined) { Some(SimpleParsed(idOpt.get, textOpt.get)) } else { None } } else { None } } } ~~~ 這是一個簡單的Jackson解析器。當我們保存，SBT會重新編譯代碼和運行測試。代碼變得越來越好了！ ~~~ info] SimpleParser should [info] + work with basic tweet [info] + reject a non-JSON tweet [info] x ignore nested content [info] '2' is not equal to '1' (SimpleParserSpec.scala:32) [info] + fail on partial content [info] == com.twitter.sample.SimpleParserSpec == ~~~ 哦。我們需要檢查嵌套對象。讓我們在token讀取循環處添加一些丑陋的守衛。 ~~~ def parse(str: String) = { val parser = parserFactory.createJsonParser(str) var nested = 0 if (parser.nextToken() == START_OBJECT) { var token = parser.nextToken() var textOpt:Option[String] = None var idOpt:Option[Long] = None while(token != null) { if (token == FIELD_NAME && nested == 0) { parser.getCurrentName() match { case "text" => { parser.nextToken() textOpt = Some(parser.getText()) } case "id" => { parser.nextToken() idOpt = Some(parser.getLongValue()) } case _ => // noop } } else if (token == START_OBJECT) { nested += 1 } else if (token == END_OBJECT) { nested -= 1 } token = parser.nextToken() } if (textOpt.isDefined && idOpt.isDefined) { Some(SimpleParsed(idOpt.get, textOpt.get)) } else { None } } else { None } } ~~~ …測試通過了！ ## 打包和發布現在我們已經可以運行package命令來生成一個jar文件。不過我們可能要與其他組分享我們的jar包。要做到這一點，我們將在StandardProject基礎上構建，這給了我們一個良好的開端。第一步是引入StandardProject為SBT插件。插件是一種為你的構建引進依賴的方式，注意不是為你的項目引入。這些依賴關系定義在project/plugins/Plugins.scala文件中。添加以下代碼到Plugins.scala文件中。 ~~~ import sbt._ class Plugins(info: ProjectInfo) extends PluginDefinition(info) { val twitterMaven = "twitter.com" at "http://maven.twttr.com/" val defaultProject = "com.twitter" % "standard-project" % "0.7.14" } ~~~ 注意我們指定了一個Maven倉庫和一個依賴。這是因為這個標準項目庫是由twitter托管的，不在SBT默認檢查的倉庫中。我們也將更新項目定義來擴展StandardProject，包括SVN發布特質，和我們希望發布的倉庫定義。修改SampleProject.scala ~~~ import sbt._ import com.twitter.sbt._ class SampleProject(info: ProjectInfo) extends StandardProject(info) with SubversionPublisher { val jackson = "org.codehaus.jackson" % "jackson-core-asl" % "1.6.1" val specs = "org.scala-tools.testing" % "specs_2.8.0" % "1.6.5" % "test" override def subversionRepository = Some("http://svn.local.twitter.com/maven/") } ~~~ 現在如果我們運行發布操作，將看到以下輸出 ~~~ [info] == deliver == IvySvn Build-Version: null IvySvn Build-DateTime: null [info] :: delivering :: com.twitter#sample;1.0-SNAPSHOT :: 1.0-SNAPSHOT :: release :: Wed Nov 24 10:26:45 PST 2010 [info] delivering ivy file to /Users/mmcbride/projects/sbt-sample/target/ivy-1.0-SNAPSHOT.xml [info] == deliver == [info] [info] == make-pom == [info] Wrote /Users/mmcbride/projects/sbt-sample/target/sample-1.0-SNAPSHOT.pom [info] == make-pom == [info] [info] == publish == [info] :: publishing :: com.twitter#sample [info] Scheduling publish to http://svn.local.twitter.com/maven/com/twitter/sample/1.0-SNAPSHOT/sample-1.0-SNAPSHOT.jar [info] published sample to com/twitter/sample/1.0-SNAPSHOT/sample-1.0-SNAPSHOT.jar [info] Scheduling publish to http://svn.local.twitter.com/maven/com/twitter/sample/1.0-SNAPSHOT/sample-1.0-SNAPSHOT.pom [info] published sample to com/twitter/sample/1.0-SNAPSHOT/sample-1.0-SNAPSHOT.pom [info] Scheduling publish to http://svn.local.twitter.com/maven/com/twitter/sample/1.0-SNAPSHOT/ivy-1.0-SNAPSHOT.xml [info] published ivy to com/twitter/sample/1.0-SNAPSHOT/ivy-1.0-SNAPSHOT.xml [info] Binary diff deleting com/twitter/sample/1.0-SNAPSHOT [info] Commit finished r977 by 'mmcbride' at Wed Nov 24 10:26:47 PST 2010 [info] Copying from com/twitter/sample/.upload to com/twitter/sample/1.0-SNAPSHOT [info] Binary diff finished : r978 by 'mmcbride' at Wed Nov 24 10:26:47 PST 2010 [info] == publish == [success] Successful. [info] [info] Total time: 4 s, completed Nov 24, 2010 10:26:47 AM ~~~ 這樣（一段時間后），就可以在?[binaries.local.twitter.com](http://binaries.local.twitter.com/maven/com/twitter/sample/1.0-SNAPSHOT/)?上看到我們發布的jar包。 ## 添加任務任務就是Scala函數。添加一個任務最簡單的方法是，在你的項目定義中引入一個val定義的任務方法，如 ~~~ lazy val print = task {log.info("a test action"); None} ~~~ 你也可以這樣加上依賴和描述 ~~~ lazy val print = task {log.info("a test action"); None}.dependsOn(compile) describedAs("prints a line after compile") ~~~ 刷新項目，并執行print操作，我們將看到以下輸出 ~~~ > print [info] [info] == print == [info] a test action [info] == print == [success] Successful. [info] [info] Total time: 0 s, completed Nov 24, 2010 11:05:12 AM > ~~~ 所以它起作用了。如果你只是在一個項目定義一個任務的話，這工作得很好。然而如果你定義的是一個插件的話，它就很不靈活了。我可能要 ~~~ lazy val print = printAction def printAction = printTask.dependsOn(compile) describedAs("prints a line after compile") def printTask = task {log.info("a test action"); None} ~~~ 這可以讓消費者覆蓋任務本身，依賴和/或任務的描述，或動作本身。大多數SBT內建的動作都遵循這種模式。作為一個例子，我們可以通過修改內置打包任務來打印當前時間戳 ~~~ lazy val printTimestamp = task { log.info("current time is " + System.currentTimeMillis); None} override def packageAction = super.packageAction.dependsOn(printTimestamp) ~~~ 有很多例子介紹了怎樣調整SBT默認的StandardProject，和如何添加自定義任務。 ## 快速參考 ### 常用命令 * actions – 顯示這個項目中可用的動作 * update – 下載依賴 * compile – 編譯源文件 * test – 運行測試 * package – 創建一個可發布的jar文件 * publish-local – 在本地ivy緩存中安裝構建好的jar包 * publish – 將你的jar推到一個遠程庫中（如果配置了的話） ### 更多命令 * test-failed – 運行所有失敗的規格測試 * test-quick – 運行任何失敗的和/或依賴更新的規格 * clean-cache – 刪除SBT緩存各種的東西。就像sbt的clean命令 * clean-lib – 刪除lib_managed下的一切 ### 項目布局待續 Built at?[@twitter](http://twitter.com/twitter)?by?[@stevej](http://twitter.com/stevej),?[@marius](http://twitter.com/marius), and?[@lahosken](http://twitter.com/lahosken)?with much help from?[@evanm](http://twitter.com/evanm),?[@sprsquish](http://twitter.com/sprsquish),?[@kevino](http://twitter.com/kevino),?[@zuercher](http://twitter.com/zuercher),?[@timtrueman](http://twitter.com/timtrueman),?[@wickman](http://twitter.com/wickman), and[@mccv](http://twitter.com/mccv); Russian translation by?[appigram](https://github.com/appigram); Chinese simple translation by?[jasonqu](https://github.com/jasonqu); Korean translation by?[enshahar](https://github.com/enshahar); Licensed under the?[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).