4/19/2013

eclipse-scala-tools 0.0.4 released!

eclipse-scala-tools is an Eclipse plug-in for sbt.

This is a list of new features and changes in this version:

  • Use system proxy setting
  • SBT 0.12 Support (SBT 0.7 and 0.10 are removed)
  • Use sbteclipse to generate Eclipse project configuration files
  • Outline view

And I have a plan about new features in the future release:

  • Scala IDE 3.x and sbt 0.13 support
  • Select some major libraries at the project creation wizard
  • Multi project support
  • Wizard to import existing sbt project from out of workspace

I hope this plug-in helps you. Enjoy!

4/04/2013

solr-scala-client 0.0.7 is now available!

solr-scala-client is a simple Apache Solr client for Scala based on SolrJ.

The list of new features in 0.0.7:

  • Add build for Scala 2.10
  • Upgrade to SolrJ 4.2.0
  • Support search result highlighting

In this entry, I introduce search result highlighting which is a new feature in this release.

How to Highlight?

You can configure the query to return the highlighted content using QueryBuilder#highlight(). The highlighted field is required, but prefix and suffix is not required. They are optional(default is <em>...</em>).

The highlighted content is set as the "highlight" property to the Map or the case class.

val result = client.query("content: Scala")
  // NOTE: unique key field is required.
  .fields("id")
  // Specify the highlighted field, prefix and postfix (prefix and postfix is optional).
  .highlight("content", "", "")
  .getResultAsMap()

result.documents.foreach { doc: Product =>
  println("id: " + doc("id"))
  println(doc("highlight")) // highlighted content is set as the "highlight" property
}

In SolrJ, we have to map retrieved documents and highlighted contents using the unique key of the index schema. solr-scala-client expects that the unique key is "id".

If your schema has the different field as the unique key, you can specify the unique key name as following:

val result = client.query("content: Scala")
  .id("documentId") // Specify the unique key name
  .fields("documentId")
  .highlight("content", "", "")
  .getResultAsMap()

2/25/2013

Accelerate Play2 development mode

Play2 development mode is very heavy because it checks file modification and reload classes for each requests. And it seems to process requests in serial. So loading web pages which contain many resources such as external CSS, JavaScript files or images is stressful.

play2-fastassets accelerates Play2 development mode by leveraging browser cache.

Replace the routing to controllers.Assets.at by jp.sf.amateras.play2.fastassets.FastAssets.get in conf/routes. This method returns a response which has a header: Cache-Control: private, max-age=3600.

#GET /assets/*file controllers.Assets.at(path="/public", file)
GET /assets/*file jp.sf.amateras.play2.fastassets.FastAssets.get(file)

And add following configurations into conf/application.conf.

fastassets.urlPath=/assets
fastassets.realPath=/public

Use FastAssets.at instead of routes.Assets.at in HTML templates.

@(title: String)(content: Html)
@import jp.sf.amateras.play2.fastassets.FastAssets
<!DOCTYPE html>
<html>
  <head>
    <title>@title</title>
    <link rel="stylesheet" media="screen" href="@FastAssets.at("stylesheets/main.css")">
    <link rel="shortcut icon" type="image/png" href="@FastAssets.at("images/favicon.png")">
    <script src="@FastAssets.at("javascripts/jquery-1.7.1.min.js")" type="text/javascript"></script>
  </head>
  <body>
    @content
  </body>
</html>

FastAssets.at appends a last modified timestamp to the filename and your browser cache it. When you update the file, this timestamp is also updated. So the browser retrieves a new file from the server instead of the cached contents.

8/25/2012

Jerkson in Play2 development mode

Play2 includes Jerkson which is a Scala wrapper for the Java based JSON library Jackson.

It's very useful and I'm loving it. However when we use it in the Play2 application, it fails to deserialize case classes after reloading in development mode. See the following code:

import com.codahale.jerkson.Json

val json = Json.generate(UserInfo("Naoki Takezoe"))
val info = Json.parse[UserInfo](json)

After reloading in the development mode, this code throws an exception such as ParsingException: Unable to find a case accessor for models.UserInfo.

The reason of this problem is Jerkson hold a class loader in the singleton object.

So this problem could be solved to create new instance which is mixed-in com.codahale.jerkson.Json trait for each Json serialization / deserialization as following:

// This code works in the development mode!
import com.codahale.jerkson.Json

val json = new Json{}.generate(UserInfo("Naoki Takezoe"))
val info = new Json{}.parse[UserInfo](json)

Some of hotswap solutions (reloading recompiled classes) on JVM have similar problem because they are based on classloader swapping. They can make rapid development on JVM and very helpful for our work. However sometimes they trouble us like this.

6/24/2012

Scalagen - A Source Code Generator for ORMs

Scalagen is a source code generator for ORMs. In the current version of Scalagen supports ScalaQuery and Anorm.

It's possible to use as a sbt plug-in. I show how to use Scalagen as a sbt plug-in for ScalaQuery. Scalagen generates table definition objects and case classes which correnspond to them.

In project/plugins.sbt, add:

resolvers += ("amateras snapshot" at "http://amateras.sourceforge.jp/mvn-snapshot/")

addSbtPlugin("jp.sf.amateras.scalagen" % "scalagen-sbtplugin" % "0.1-SNAPSHOT")

libraryDependencies ++= Seq(
  // for ScalaQuery
  "jp.sf.amateras.scalagen" %% "scalagen-scalaquery" % "0.1-SNAPSHOT",
  // for Anorm
  //"jp.sf.amateras.scalagen" %% "scalagen-anorm" % "0.1-SNAPSHOT",
  // JDBC driver for your database
  "org.hsqldb" % "hsqldb" % "2.2.8"
)

In build.sbt, add:

seq(jp.sf.amateras.scalagen.ScalagenPlugin.scalagenSettings: _*)

scalagenConfiguration := jp.sf.amateras.scalagen.Settings(
  // for ScalaQuery
  generator = new jp.sf.amateras.scalagen.ScalaQueryGenerator(),
  // for Anorm
  //generator = new jp.sf.amateras.scalagen.ScalaQueryGenerator(),
  driver = "org.hsqldb.jdbcDriver",
  url = "jdbc:hsqldb:hsql://localhost/",
  username = "sa",
  password = "",
  catalog = null,
  schemaPattern = null,
  tablePattern = null
)

Execute sbt scalagen, source files are generated into src/main/scala/models.

See more details about Scalagen at the following URL:
https://github.com/takezoe/scalagen

5/27/2012

solr-scala-client 0.0.2 is now available!

solr-scala-client is a Apache Solr client for Scala wrapping SolrJ.

The list of new features in 0.0.2:

  • Added initializer which configures SolrClient.
  • Added basic authentication support as initializer.
  • Added facet search support.
  • Added case class support as query results and query parameters.

The query result became to be returned as MapQueryResult or CaseClassQueryResult instead of List[Map[String, Any]]. This object contains both of documents and facet counts.

val result = client.query("name:%name%")
      .fields("id", "manu", "name")
      .facetFields("manu")
      .sortBy("id", Order.asc)
      .getResultAsMap(Map("name" -> "ThinkPad X201s"))

// retreive documents
result.documents.foreach { doc =>
  println("id: " + doc("id"))
  println("  manu: " + doc("manu"))
  println("  name: " + doc("name"))
}

// retreive facet counts
result.facetFields.foreach { case (field, counts) =>
  println("field: " + field)
  counts.foreach { case (manu, count) =>
    println("  " + manu + ": " + count)
  }
}

It's possible to use the case class as the query result and the query parameter instead of Map[String, Any]. Note: update operations don't support the case class in 0.0.2. It will be supported in the next version.

// the case class for the document
case class Product(id: String, manu: Option[String], name: String)
// the case class for the parameter
case class Param(name: String)

// query using case classes
val result = client.query("name:%name%")
      .fields("id", "manu", "name")
      .facetFields("manu")
      .sortBy("id", Order.asc)
      .getResultAs[Product](Param("ThinkPad"))

result.documents.foreach { product =>
  println(product)
}

As small improvement of SolrClient, it became to accept the initializer function. This function takes CommonsHttpSolrServer and can do any processing for it.

val client = new SolrClient("http://localhost:8983/solr", {
  server: CommonsHttpSolrServer => // initialize...
})

0.0.2 contains the BASIC authentication support as the initializer. see the following example.

val client = new SolrClient("http://localhost:8983/solr", 
  Auth.basic("username", "password"))

I think solr-scala-client does not have enough features to use in production yet. Therefore I will improve it through use in my project.

4/29/2012

The simple Apache Solr client for Scala

I pushed solr-scala-client into github. This is a simple Apache Solr client for Scala wrapping SolrJ.

The basic concept of solr-scala-client is providing fluent interface and wrapping SolrJ classes by Scala collection API. This is an example to register documents into the Solr server which is working at localhost:8983.

import jp.sf.amateras.solr.scala._

val client = new SolrClient("http://localhost:8983/solr")

client
  .add(Map("id"->"1", "name" -> "ThinkPad X201s"))
  .add(Map("id"->"2", "name" -> "ThinkPad X220"))
  .add(Map("id"->"3", "name" -> "ThinkPad X121e"))
  .commit

add() takes a variable-length argument. So an example above could be rewritten as follows:

client.add(
  Map("id"->"1", "name" -> "ThinkPad X201s"),
  Map("id"->"2", "name" -> "ThinkPad X220"),
  Map("id"->"3", "name" -> "ThinkPad X121e")
).commit

Next, see the following example to search document using the query.

// query
val result: List[Map[String, Any]] =
  client.query("name:%name%")
    .fields("id", "manu", "name")
    .sortBy("id", Order.asc)
    .getResult(Map("name" -> "ThinkPad"))

result.foreach { doc =>
  println("id: " + doc("id"))
  println("name: " + doc("name"))
  println("--")
}

%VARNAME% in the query is replaced by given parameters as Map and the result is returned as List[Map[String, Any]].

The current version of solr-scala-client does not support facet search yet. I wish to support it in the near future. And I'm also planning about using case class to specify document or parameters instead of Map.