CloudFoundry and a "normal" WAR - How to bind to a datasource on CloudFoundry

CloudFoundry and a "normal" WAR - How to bind to a datasource on CloudFoundry

so here's an interesting one; really want to use "cloud" and i have existing non-Spring WAR.  how can we bind to a datasource in a "flexible" way that doesn't require too much coding or rebuilding as a Spring App?

1. import a package from Spring (it looks like we're adding Spring, but there's really no influence, just one class)

 
  
   org.springframework.maven.milestone
   Spring Maven Milestone Repository
   http://repo.spring.io/milestone
  
 


  
   org.springframework.cloud
   cloudfoundry-connector
   0.9.5
  
  
   org.springframework.cloud
   spring-service-connector
   0.9.5
  

2. provide a way to bind to a CloudFoundry datasource

package io.pivotal.poc.simple.service;

import javax.sql.DataSource;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.support.DefaultListableBeanFactory;
import org.springframework.cloud.CloudException;
import org.springframework.cloud.config.java.AbstractCloudConfig;

public class CloudConfiguration extends AbstractCloudConfig {

 private static final Logger logger = Logger.getLogger(CloudConfiguration.class);
 
 public CloudConfiguration() {
  BeanFactory factory = new DefaultListableBeanFactory();
  try {
   this.setBeanFactory(factory);
  }
  catch (CloudException e) {
   logger.warn("no cloud",e);//TODO clean up message
  }
 }
 
 public DataSource inventoryDataSource() {
  ServiceConnectionFactory connectionFactory = connectionFactory();
  if (connectionFactory != null) {
   return new ServiceConnectionFactory().dataSource("twitter-pgsql");//TODO clean up hard coding
  }//end if
  return null;
 }
 
}

3. provide the datasource as a servlet context attribute

package io.pivotal.poc.simple.listener;

import io.pivotal.poc.simple.service.CloudConfiguration;

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
import javax.sql.DataSource;

import org.apache.commons.dbcp.BasicDataSource;
import org.apache.log4j.Logger;

@WebListener
public class DatasourceListener implements ServletContextListener {
 
 private static final Logger logger = Logger.getLogger(DatasourceListener.class);

 private CloudConfiguration cloudConfiguration = new CloudConfiguration();
 
 public static final String DATASOURCE_ATTRIBUTE = "dataSource";
 
 @Override
 public void contextInitialized(ServletContextEvent sce) {
  if (!bindCloudDataSource(sce)) {
   logger.warn("not running in the cloud");;
   bindLocalDataSource(sce);
  }//end if
 }

 @Override
 public void contextDestroyed(ServletContextEvent sce) { }

 //creates a "local" (in-memory) binding for testing purposes
 private void bindLocalDataSource(ServletContextEvent sce) {
  //create a database
  BasicDataSource factory = new BasicDataSource();
  factory.setDriverClassName("org.h2.Driver");//TODO - remove hardcoding
  factory.setUrl("jdbc:h2:mem:a");
  factory.setUsername("sa");
  factory.setPassword("");
  //set
  sce.getServletContext().setAttribute(DATASOURCE_ATTRIBUTE, factory);
 }
 
 //runs the cloud configuration component
 private boolean bindCloudDataSource(ServletContextEvent sce) {
  DataSource dataSource = cloudConfiguration.inventoryDataSource();
  if (dataSource != null) {
   sce.getServletContext().setAttribute(DATASOURCE_ATTRIBUTE, dataSource);
   return true;
  }//end if
  return false;
 }
}

3. you're done

a couple of assumptions;

- not using a Dependency Injection container (at all)

- using servlets/etc to pull a datasource object from the servlet context

caveat; this is very crude and will be refined as it gets tested out

Spring Batch - Parallel Item Writer

Spring Batch - Parallel Item Writer

here's a quick solution i put together for stackoverflow and writing to many itemwriters in parallel

here's the class

package de.incompleteco.spring.batch.item.support;

import java.util.List;

import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.CompositeItemWriter;
import org.springframework.core.task.TaskExecutor;
import org.springframework.util.Assert;

import de.incompleteco.spring.domain.SimpleEntity;

public class ParallelCompositeItemWriter extends CompositeItemWriter< SimpleEntity > {

 private List< ItemWriter< ? super SimpleEntity > > delegates;
 
 private TaskExecutor taskExecutor;
 
 @Override
 public void write(final List< ? extends SimpleEntity > item) throws Exception {
  for (final ItemWriter< ? super SimpleEntity > writer : delegates) {
   taskExecutor.execute(new Runnable()  {
    @Override
    public void run() {
     try {
      writer.write(item);
     } catch (Throwable t) {
      rethrow(t);
     } 
    }
    
    private void rethrow(Throwable t) {
     if (t instanceof RuntimeException) {
      throw (RuntimeException) t;
     }
     else if (t instanceof Error) {
      throw (Error) t;
     }
     throw new IllegalStateException(t);
    }  
   });
  }//end for
 }


 public void setTaskExecutor(TaskExecutor taskExecutor) {
  this.taskExecutor = taskExecutor;
 }
 
 @Override
 public void setDelegates(List < ItemWriter < ? super SimpleEntity > > delegates) {
  this.delegates = delegates;
  super.setDelegates(delegates);
 }

 @Override
 public void afterPropertiesSet() throws Exception {
  super.afterPropertiesSet();
  Assert.notNull(taskExecutor,"Task executor needs to be set");
 }
 
}

and here's a sample configuration to run it

 
  
   
    
   
  
 

 
  
 
 
 
  
  
 
 
 
  
   
   
   
    
   
  
  
   
   
   
    
   
    
 
 
 
  
   
  
  
   
    
  
   
    
 
 
 
 

 
     
     
     
     
     
         
             jdbc:h2:mem:a;DB_CLOSE_DELAY=-1
         
     
  
 
 
     
     
     
     
     
         
             jdbc:h2:mem:b;DB_CLOSE_DELAY=-1
         
     
   
 
 
  
 
 
 
  
 
 

 
 
 

 
     
     
 

there's a couple of things to note;
- Bitronix JTA is used to support XA transactions across multiple databases
- the jdbc stuff is pretty rubbish and needs to be done better

Schemaless PostgreSQL

Schemaless PostgreSQL

excellent presentation on Schemaless PostgreSQL

https://wiki.postgresql.org/images/b/b4/Pg-as-nosql-pgday-fosdem-2013.pdf

Spring LocalContainerEntityManagerFactoryBean, packagesToScan and JTA

Spring LocalContainerEntityManagerFactoryBean, packagesToScan and JTA

here's an interesting one, Hibernate 4, Spring JPA (using LocalContainerEntityManagerFactoryBean) and JTA.  the scenario is, testing in JUnit and need to specify (and emulate) a JTA transaction. unfortunately, it's not all straight out-of-the-box, but here it is;

first, LocalContainerEntityManagerFactoryBean treats everything as a non-JTA transaction, so there's a little extra configuration to be done.  To switch it out, Spring does provide a postProcessor for the PersistenceUnit and a mutable persistent unit to it, but we have to write our own switch.  (it's pretty straight forward, but maybe we can ask the nice chaps to include it out of the box in 3.2.x?)

here's my cut - you can roll/cut & paste your own

package org.incompletecode.spring.batch.jpa;
 
import org.springframework.orm.jpa.persistenceunit.MutablePersistenceUnitInfo;
import org.springframework.orm.jpa.persistenceunit.PersistenceUnitPostProcessor;
 
public class JTAPersistenceUnitPostProcessor implements
  PersistenceUnitPostProcessor {
 
 @Override
 public void postProcessPersistenceUnitInfo(MutablePersistenceUnitInfo pui) {
  //switch the nonJTA to JTA
  pui.setJtaDataSource(pui.getNonJtaDataSource());
  pui.setNonJtaDataSource(null);
 }
 
}

and now here's a quick example of using Bitronix as your JTA provider

 
 
 
 
     
 
  
 
 
 
     
 
 
 
     
     
 
 
 
 
 
      
      
      
      
      
      
          
              jdbc:h2:mem:a;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1
          
        
 
 

Groovy DDLUtils by Scott Frederick

Groovy DDLUtils by Scott Frederick

DDLUtils is a great little apache project that really can help standardize and minimize the impact of databases in projects from your in-memory database to production.  But, Scott Frederick has made it a little more accessible with his Groovy DDLUtils.

Check out his blog post here

http://scottfrederick.blogspot.com/2011/11/groovy-ddlutils.html


and the project here


https://github.com/scottfrederick/groovy-ddlutils


the base DDLUtils is here


http://db.apache.org/ddlutils/