issuekey,title,description,storypoint
XD-1,"HDFS ItemWriter","Base integration of core HDFS writer functionality with Spring Batch.",1
XD-2,"HDFS Core writing helper classes","Simple file writer that has existed in the spring hadoop samples.",1
XD-6,"Channel Registry",NULL,3
XD-7,"Tuple data structure","The tuple data structure should be backward compatible in functionality for use in spring batch. Porting over FieldSet tests in spring batch to use the tuple data structure is one way help ensure that compatibility.",1
XD-8,"Syslog Ingestion","Have a syslog.xml config file that can be added to a module and registered with a module registry.",1
XD-9,"Basic implementation of a reactor based tcp server",NULL,13
XD-10,"Reactor based http ingestion","When there is support for boostrapping a http server in the reactor project, and inbound SI adapter and associated XD source module should be created.",5
XD-11,"Initial work that uses the module architecture from DIRT ",NULL,5
XD-12,"Jolokia based aggregator for cluster monitoring",NULL,8
XD-13,"Tail file channel adapters",NULL,8
XD-14,"Design for deploying XD on EC2","create enough of a design to develop additional stories.",3
XD-18,"Websocket based taps",NULL,5
XD-20,"DIRT Runtime that deploys an application context across multiple nodes using redis.",NULL,3
XD-23,"add file source and sink modules",NULL,1
XD-24,"Create pipes and filters DSL for ingestion","Initial simple handcoded implementation for straight through pipe and filter model, e.g. a | b | c",2
XD-26,"Basic Performance Test For syslog injestion",NULL,20
XD-27,"Gemfire CQ module for ingestion",NULL,1
XD-28,"Create simple gague service","A gauge just stores a number. Implementations for in-memory and redis.",1
XD-29,"Create rich gauge service","A rich gauge stores a number and also rmd, min, max. Implementations for in-memory and redis.",5
XD-30,"Create a simple counter service","A simple counters can increment/decrement a number. Implementations for in-memory and redis.",1
XD-31,"Create field-value counters","A field-value counter is useful for bar chart graphs, Strings on x-axis and count on y-axis. Maps well to zset in redis. Implementations for in-memory and redis.",5
XD-35,"Create CI process for XD build","bamboo based",1
XD-36,"SI Outbound HDFS Channel Adapter",NULL,3
XD-37,"Gradle based multi-project build","multi project build. - look to Spring Framework for source of starting point.",2
XD-40,"Build script that creates an executable server as an artifact","Gradle application plugin is a good starting point. this should be the main server that would host SI based modules to do syslog->file ingestion (as an example)",2
XD-41,"Add JUnit @Rule so Tests Fail Fast with Clear Messaging if Redis Not Available",NULL,1
XD-43,"Metric repositories should support Spring Data CrudRepository interface","This provides common CRUD behavior and a shared interface that can be useful in testing scenarios. ",5
XD-44,"Redis based repositories should use a NamingStrategy class to calculate the name of the key to use for persistence","RedisCounterRepository and RedisGaugeRepository have duplicated code that needs to be factored out into a one place. One such duplication is the determination of the key name to use for persistence. This should be abstracted out into a strategy helper class.",1
XD-45,"Remove the expiry of keys in Redis based repositories","There is duplicated code in Redis based repositories that related to expiry behavior, move into a common shared helper class and/or base class.",1
XD-47,"HDFS sink module",NULL,1
XD-48,"Parameterizable streams",NULL,2
XD-49,"Move Redis Queue Channel Adapters into spring-integration-redis","Currently these implementations are in the spring-xd-dirt module, but they should be moved into spring-integration-redis. We are already depending upon Spring Integration 3.0 snapshots since the ChannelRegistry implementation is not yet at a milestone, so this should be okay for the Redis Channel Adapters also - until Spring Integration M2 is released.",3
XD-50,"Add tap support to DIRT","syntax: {code} tap @ somechannel --key=value | somecounter {code} ",2
XD-51,"Add xd.stream.name property in StreamPlugin",NULL,1
XD-52,"add twitter search source module",NULL,1
XD-53,"Design and document desired high level DSL for configuring data processing in XD","Start to explore how the DSL can cover both advanced (non-linear) spring integration flows as well as spring batch jobs.",13
XD-54,"XD Metrics backed Message Counter","A Spring Integration based @ServiceActivator that counts the number of messages using the Spring XD metrics support",1
XD-55,"SI ServiceActivator for an XD Metrics backed Field Value Counter","A Spring Integration based @ServiceActivator that counts the occurrence of field names, from either a tuple data structure or a POJO, using the Spring XD metrics support.",3
XD-56,"Switch to use Lettuce driver for Redis","Replace the use of Jedis with Lettuce as it has higher performance",2
XD-57,"add counter module",NULL,1
XD-58,"build.gradle doesn't handle a small handful of libraries","Trying to build spring-xd for the first resulted in lots of errors inside STS (I had an empty .m2 repo).",1
XD-59,"Tuple should support storing nested tuples","Nested tuple structures shoudl be supported, getTuple(int index), getTuple(String name)",1
XD-60,"Saving a metric (Counter, Gauge..) with an existing name should throw an exception","The difference between saving a new metric and updating an existing one needs to be defined. Suggest that if we try to save when an existing counter is already in the database to throw exception, such as DataIntegrityViolationException.",1
XD-61,"Create distributable artifact that contains server application and start/stop scripts","The gradle application task should get us most of the way to create a distributable artifact akin to what you see when downloading tomcat/jetty etc. Now there is a launch task task(launch, dependsOn: 'classes', type: JavaExec) { main = 'org.springframework.xd.dirt.stream.StreamServer' classpath = sourceSets.test.runtimeClasspath } The same main should be referenced in the application plugin, a task to create a .zip distributable is needed. Ideally would be nice to 1. download .zip 2. unzip 3. cd spring-xd/bin 4. xdserver start and gracefully shutdown later with 5. xdserver stop I don't know if we can/should bundle redis, I think we should bundle it. The scripts can be for unix/linux and for windows. Discuss a brew based install as well. ",8
XD-62,"Use the tuple data structure to process data in a spring batch step ","Do not require a POJO in order to do end-to-end processing in a batch step.",5
XD-63,"Document tuple data structure on XD wiki",NULL,2
XD-65,"Gemfire Sink to update a gemfire cache.","Update a gemfire region.",2
XD-66,"Gauge and Counter hash and equals should not depend on values","The value is mutable so this causes problems if storing metrics in a HashSet, for example.",1
XD-67,"Submit a brew-based install for Spring XD","- Host the Spring XD distributable zip somewhere that is accessible by external http request. - Create brew formula for Spring XD install while specifying redis as dependency. - starting up stream server upon successful brew install couple of questions: - should we name the brew task springxd? (name not taken yet) - should we start the stream server as part of the brew install process? - should we specify redis as a recommended dependency? user can pass in 'brew install springxd --without-redis' to skip redis installation. by default, 'brew install springxd' will install redis as well.",8
XD-68,"Export of data from HDFS to a relational database","Based on a single process running a Spring Batch job, support the ETL of data from HDFS to a RDBMS",8
XD-69,"Export of data from HDFS to MongoDB","Based on a single process Spring Batch job, ETL of data from HDFS to MongoDB.",5
XD-70,"Create general structure for AsciiDoc based wiki and Spring XD guide.","Adopt Asciidoc as the markdown syntax, useful for generating pdf and more feature rich than standard github flavored markdown. Loosely following the conventions of https://community.jboss.org/wiki/TheHolyGrailAsciiDocOnGitHubToDocBookTrain that have generate docbook/pdf docs from the Asciidoc wiki. The asciidoctor project is a key element in the adoption of AsciiDoc for use as the format in github, it is the rendering engine used by github for AsciiDoc. See http://asciidoctor.org/docs/asciidoc-writers-guide/ for guidance. ",2
XD-71,"Remove UUID from Tuple class or replace with more efficient implementation","The Java UUID class is known not to be the fasted implementation available. See https://github.com/stephenc/eaio-uuid and http://mvnrepository.com/artifact/com.eaio.uuid/uuid for high perf impls. ",1
XD-72,"Provide a http source","stream should be able to ingest data from http ",5
XD-73,"add test to start/stop stream server","in additional to existing tests that check for redis connection, we need to add tests that start/stop stream server. ",2
XD-74,"Create XD module for tail file adapter",NULL,1
XD-75,"Add redis bundle to distribution zip file","for linux and mac",5
XD-76,"Add gemfire-server application to the distribution zip of the project spring-xd-gemfire-server",NULL,1
XD-78,"Documentation for ""http | file"" processing","Put on the guide as a section in an 'input sources' wiki page. https://github.com/springsource/spring-xd/wiki/GuideGettingStarted ",1
XD-79,"End user guide for data streams","Put on the guide as a section in an 'streams' wiki page. End user focused, no need to mention spring underpinning, impl details. ",5
XD-80,"Documentation for ""http | hdfs"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",1
XD-81,"Documentation for ""syslog | file"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-82,"Documentation for ""tail | file"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-83,"Documentation for ""tail | hdfs"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",1
XD-84,"Documentation for ""http | gemfire"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-85,"Documentation for ""gemfirecq | file"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-86,"Documentation for ""twittersearch | file"" processing","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-87,"Documentation that introduces taps","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-88,"Documentation for counter taps","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-89,"Documentation for gauge taps","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-90,"Documentation for rich gauge taps","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-91,"Documentation for field value taps","Put on the guide as a section in an 'input-stream' wiki page. ",3
XD-93,"Normalize and refactor component packaging decomposition","Normalize and refactor as needed functionality currently included in - spring-integration-core (LocalChannelRegistry) - spring-integration-module (Module types upon which Flow are built) - xd-module (Depend Module types common to DIRT and Spring Integration) - spring-integration-flow (Flow specific types, namespace support, etc)",5
XD-94,"Document The ability to use flows in streams","Test and document e.g. ""http | flow1 | flow2 | file""",3
XD-95,"The ability to override default module path for the plugin or an individual flow","Currently modules are assumed to be in classpath:/META-INF/spring/integration/module/${flow}.xml. To reuse modules defined with DIRT may require more flexibility. e.g., ",3
XD-96,"The ability to import beans referenced in the main context into a module","This should be a core feature of any Spring based module. The plugin should be able to import explicitly referenced beans. This minimizes potential side effects of making the module a child context and is simpler than declaring a shared context (parent) of the application and the module. Provide namespace support for flow:",3
XD-97,"The ability to route within streams","A Flow or a processor component may require routing semantics. Currently the stream assumes a single input and output for each module. A Flow may support multiple outputs - Switch routing that is - Recipient list is not currently supported (another subtask?). We need to support semantics like: a |[output.foo:c,output.bar:d,default:e]",8
XD-98,"Documentation on the module system and how to contribute new modules","For people who are familiar with Spring/Spring Integration provide documents that show how to add additional input sources/sinks.",3
XD-99,"Sonar build is failing","https://build.springsource.org/browse/XD-SONAR-34 Caused by: java.lang.ClassNotFoundException: org.sonar.api.Plugin at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50) at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244) at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230) ... 94 more",1
XD-100,"Rename Tuple class in spring-xd-tuple","The Tuple classes in Reactor follow the more traditional data structure concept of Tuples, an immutable fixed length sequence of values where each value can have different types. They are ordered and can often be access by index. An example in a static language is the Tuple class found in .NET http://msdn.microsoft.com/en-us/library/system.tuple.aspx or in Scala http://www.tutorialspoint.com/scala/scala_tuples.htm Using this standard definition of a Tuple, they do not support named values. There is also a different tuple class instance for each length, e.g. Tuple, Tuple. The Tuple class in XD is more like a record or named tuple. Python has a named tuple concept - http://docs.python.org/2/library/collections.html#collections.namedtuple and http://stackoverflow.com/questions/1490413/languages-that-allow-named-tuples shows that other languages use the term 'Record' for a 'named tuple' - Haskell, Standard ML, OCaml, and F#. http://en.wikibooks.org/wiki/F_Sharp_Programming/Tuples_and_Records#Defining_Records So boiling it all down, to avoid conflicts of names, and also to open up the possibility of using Reactor tuples as keys (instead of strings for names), we should change the name to either NamedTuple or Record. ATM, there is no direct relationship between Reactor's Tuple and NamedTuple (such as inheritance) and so probably Record is the way to go. ",2
XD-101,"Add gradle tasks that build and bundle the redis server",NULL,2
XD-102,"Create XDContainer class to start stream server","Provide optional command line arg to embed the container launcher, aka - xd-admin server. XDContainer.sh --embeddAdmin",1
XD-103,"Create XDAdmin server to start container launcher","This will launch the RedisContainerLauncher, in future will be able to select from a variety of middleware options.",1
XD-104,"Add README to be included in root directory of distribution","should explain basic layout of the distribution",1
XD-105,"Add LICENSE to be included in root directory of distribution","should contain apache licence",1
XD-106,"Container server does not log a message that it has started or stopped successfully","$ ./xd-container processing module 'Module [name=file, type=sink]' from group 'tailtest' with index: 1 processing module 'Module [name=tail, type=source]' from group 'tailtest' with index: 0 Logging of 'processing module' should have log level, time..",1
XD-107,"Clean shutdown of redis in xd-admin","A ctrl-c of xd-admin results in exception messages about disconnecting from redis. 14:16:07,327 ERROR task-scheduler-1 handler.LoggingHandler:136 - org.springframework.data.redis.RedisSystemException: Redis command interrupted; nested exception is com.lambdaworks.redis.RedisCommandInterruptedException: Command interrupted ",2
XD-108,"Build script should not package 'spring-xd-dirt' scripts ","We are packaging separate scripts to start XDAdmin and XDContainer. The Gradle application plugin will generate an unwanted 'spring-xd-dirt' scripts, this should be removed from the bin directory when creating a distribution zip.",1
XD-109,"Documentation for starting Spring XD servers",NULL,1
XD-110,"Remove use of system property xd.home to define location for install location, rely on environment variable XD_HOME","This is in the AdminMain and ContainerMain. Can get the environment property directly in java code unless provided explicitly on the command line using --xdHomeDir.",1
XD-111,"Create final distribution zip across multiple projects","The final directory structure should look like /xd /redis /gemfire inside the XD directory /xd/bin - which has xd-container and xd-admin scripts /xd/lib inside the gemfire directory /gemfire/bin - has the gemfire-server script /gemfire/lib inside the redis directory is /redis/redis-latest-v.x.y.z.tar /redis/README /readis/install-redis - script that does the basic 4 commands to install redis. There should be a gradle task that runs after the distZip task, that will take the contents of different project directories, script diretories and 'redis-binary' directories and creates the final layout for the distribution.",5
XD-112,"Update XD to Use SI 3.0.0.M2",NULL,1
XD-114,"Add install script for Redis","This assumes the redis source tar is available under $rootDir/redis/redis-2.6.13.tar.gz The install script does the following: - Check the platform OS & arch - unzip the tar, compile the sources",2
XD-115,"add SpEL 'transform' processor","It should provide an 'expression' param for SpEL and have a default pass-thru of the payload.",1
XD-116,"add SpEL 'filter' processor","It should provide an 'expression' param for SpEL and have a default value of true (accept everything).",1
XD-117,"add spring-integration-groovy to container dependencies","This will enable the use of groovy scripts within modules.",1
XD-118,"replace testsource with time and testsink with log","This should facilitate testing while avoiding any class dependencies. Also, log is a generally useful sink by itself and time is a more interesting source for testing (should accept --interval for the seconds between time messages).",1
XD-119,"HDFS sink should default to hdfs://localhost:8020","The current default is hdfs://localhost:9000 but most new distributions/installs use 8020",1
XD-120,"Find and eliminate package-level cycles across XD projects",NULL,3
XD-121,"Add pre-compiled redis distributions for the selective OS platforms",NULL,10
XD-122,"XD scripts need to have spring-integration milestone versions updated","Spring-integration version is changed to 3.0.0.M2 and since we manually create the XD scripts, they still point to the 3.0.0.BUILD-SNAPSHOT version. As discussed, we also need to have a better strategy on updating the lib directory inside the XD scripts.",2
XD-123,"XD scripts lib path needs to be dynamic","We currently have the manually created XD scripts. This makes it difficult to maintain as the lib path is error prone with the changes. We need to make sure that the properties such as lib path etc., are dynamically updated.",3
XD-124,"Clean shutdown of redis in xd-container","Need to shutdown cleanly, no exception messages are shown. Order of components in the stream should be shut down from 'first to last' (opposite of creation)",2
XD-125,"Stream documentation review",NULL,2
XD-126,"Documentation for sources, sinks, modules should define which attributes are required and which optional","This will eventually be supplied by the admin server, but for now write it up by hand in the documentation",2
XD-127,"Create TCP source module","Based off SI tcp inbound adapter. This will allow for event forwarding that can select among the existing SI serialized/deserializer options. ",3
XD-128,"Create TCP sink module","Based off SI tcp inbound adapter. This will allow for event fowarding.",3
XD-129,"Documenation for building/starting redis servers",NULL,2
XD-130,"Remove container entry in Redis when the application context event to shutdown the container is fired",NULL,3
XD-131,"Upgrade Lettuce to 2.3.2",NULL,2
XD-132,"Profile support for modules","To allow for groups of beans to be defined or not in the container that runs a module. When deploying a stream (e.g. via the REST API), it should be possible to also provide profile names. Then those would apply to any modules within that particular stream deployment.",8
XD-133,"Fail Sonar CI build if there are any package tangles violated.","Similar to what would show up on structure101 reports.",1
XD-134,"Investigate link checking tool for user guide","Asciidoc/doctor might have one as part of it toolchain",2
XD-135,"Documentation on XD Architecture","Show overall flow of data in a stream, the server components 'admin' and 'container'. How modules are deployed.",4
XD-136,"Documentation that points on how to install hadoop","Pointers to other documentation on how to install hadoop. ",3
XD-137,"Release Spring XD 1.0 M1",NULL,5
XD-138,"Prepare blog post for M1",NULL,5
XD-139,"Update README.txt to include instructions on how to build","Building XD should not be part of the out first out of the box experience, but we should include some instructions on what targets are available, such as distXD.",2
XD-140,"Parameterize syslog Source; Add Support for TCP","The syslog source currently is hard-coded to use udp on port 11111. Need to parameterize the port and provide an option to use TCP.",2
XD-141,"install-redis script should not use relative path to determine redis source dist","Currently, the install-redis script uses relative path to determine redis source dist file. Since this is error prone, we need to fix it.",1
XD-142,"StreamServer Context Lifecycle Issues","The {{ModuleDeployer}} calls {{getBeansOfType}} before the context has had its {{PropertySourcesPlaceholderConfigurer}} attached. This can cause issues with {{FactoryBean}} s with placeholders in constructor args because the unresolved placeholder is used when the {{FactoryBean}} is pre-instantiated to determine the type of object it will serve up.",8
XD-143,"Create externalized property file to support connectivity to redis","We need to have an externalized property file(under xd/conf/) for the xd-container & admin scripts to use as options. ",3
XD-147,"Remove use of application plugin for redis project","Currently redis project uses application plugin to bundle distribution. This also includes 'java plugin' which causes java specific build behavior on this project. We should try removing the use of application plugin and use something similar or custom tasks that does the bundling. ",2
XD-148,"Improve User Experience when Redis is not running","Redis is not running we get a nasty stacktrace: {code} ~/dev/git/spring-xd/dist/spring-xd/xd/bin (master)] ➔ ./xd-container 13/05/29 16:17:15 INFO support.ClassPathXmlApplicationContext: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@851052d: startup date [Wed May 29 16:17:15 EDT 2013]; root of context hierarchy 13/05/29 16:17:16 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [META-INF/spring/launcher.xml] 13/05/29 16:17:16 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [META-INF/spring/redis.xml] 13/05/29 16:17:17 INFO support.DefaultListableBeanFactory: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4934ce4a: defining beans [org.springframework.context.support.PropertySourcesPlaceholderConfigurer#0,redisConnectionFactory,org.springframework.xd.dirt.launcher.RedisContainerLauncher#0]; root of factory hierarchy 13/05/29 16:17:17 INFO support.DefaultListableBeanFactory: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4934ce4a: defining beans [org.springframework.context.support.PropertySourcesPlaceholderConfigurer#0,redisConnectionFactory,org.springframework.xd.dirt.launcher.RedisContainerLauncher#0]; root of factory hierarchy Exception in thread ""main"" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'redisConnectionFactory' defined in class path resource [META-INF/spring/redis.xml]: Invocation of init method failed; nested exception is com.lambdaworks.redis.RedisException: Unable to connect org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1488) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:524) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:461) org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295) org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223) org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292) org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:626) org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932) org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479) org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139) org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:83) org.springframework.xd.dirt.launcher.RedisContainerLauncher.main(RedisContainerLauncher.java:68) org.springframework.xd.ContainerMain.main(ContainerMain.java:68) Caused by: com.lambdaworks.redis.RedisException: Unable to connect com.lambdaworks.redis.RedisClient.connect(RedisClient.java:176) com.lambdaworks.redis.RedisClient.connectAsync(RedisClient.java:139) org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.initConnection(LettuceConnectionFactory.java:108) org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.afterPropertiesSet(LettuceConnectionFactory.java:86) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1547) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1485) ... 13 more Caused by: java.net.ConnectException: Connection refused: localhost/127.0.0.1:6379 sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599) org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) java.lang.Thread.run(Thread.java:680) {code} I think it would be helpful to provide users with some helpful advice e.g.: Redis does not seem to be running at 'localhost/127.0.0.1:6379'. Did you start or install Redis? Please see for further help http://foo/bar ",2
XD-149,"Create asciidoc toolchain script to create a 'toc2' style html output","Publishing an html version of the guide that uses the 'toc2' style format, table of contents on the left. Looks like a 'stylesheet factory' http://asciidoctor.org/docs/produce-custom-themes-using-asciidoctor-stylesheet-factory/ needs to be installed. From the theme showcase, http://themes.asciidoctor.org/preview/, the 'golo' theme has a toc2 style. In the root of the git repo for the wiki is a build.gradle file that uses the asciidoctor gradle plugin, but it doesn't support using a single file with import as input. (See See https://github.com/asciidoctor/asciidoctor-gradle-plugin/issues/15 ) The mvn plugin does support this, so maybe using mvn is an option or just a bash script.",4
XD-150,"Publish Spring XD final distribution zip as part of Bamboo artifactory plugin","Currently, Bamboo's gradle artifactory plugin has the artifacts configured to projects target(build) directory 'archives'. We need to have a way to set the final distribution archive as one of the gradle 'configurations' in our build.gradle and refer it inside bamboo artifacts.",2
XD-151,"Add Redis binaries for Windows","Presently, Spring XD does not ship Windows binaries for Redis. However, Microsoft is actively working [1] on supporting Redis on Windows. You can download Windows Redis binaries from: https://github.com/MSOpenTech/redis/tree/2.6/bin/release [1] http://blogs.msdn.com/b/interoperability/archive/2013/04/22/redis-on-windows-stable-and-reliable.aspx",2
XD-152,"Create rich gauge module","Spring config for rich gauge plus message handler to coerce a numeric or string payload to a double.",5
XD-153,"create a gauge module","Spring config for simple gauge plus message handler to process message. There is some code common to RichGaugeHandler to coerce the payload to a double that should be refactored for reuse.",5
XD-154,"Provided console output of started server","Shouldn't we have something like a ContextRefreshedEvent Listener and output some informational messages to the console, so the user knows the Container is up (Which contexts. Maybe even print a link to the docs))? Maybe even some simple ascii art (for demos)? Right now it looks somewhat barren. Redis provides something similar. This may even go hand in hand to provided a better configuration model (storing common config parameters centrally) {code} _____ _ __ _______ / ____| (_) \ \ / / __ \ | (___ _ __ _ __ _ _ __ __ _ \ V /| | | | \___ \| '_ \| '__| | '_ \ / _` | > < | | | | ____) | |_) | | | | | | | (_| | / . \| |__| | |_____/| .__/|_| |_|_| |_|\__, | /_/ \_\_____/ | | __/ | |_| v1.0.0.M1 |___/ eXtreme Data Using Redis at localhost:6379 The Server (PID: 12345) is now ready on http://myserver:123/streams Documentation: https://github.com/SpringSource/spring-xd/wiki {code}",2
XD-155,"Add a groovy script processor module","A processor module that accepts either the location of a groovy script resource or an inline script (string). Also some discussion about a default classpath location for scripts. ",5
XD-156,"Create config support for Redis","We would like to have Redis driven from a config property file under XD_HOME.",2
XD-157,"Verify use of JMX managed bean to shutdown cleanly the xd-admin and xd-container servers","In both cases there are multiple application contexts that can be running in the process. The JMX Managed bean should call 'close' on those application contexts. The more detailed lifecycle of cleanly shutting down components within those application contexts is another story.",4
XD-158,"Expose shutdown operation over http","This will allow for a simple way to shutdown the server via an HTTP call. Support for security is a separate story. The end goal is to have some shell scripts distributed that can issue HTTP requests to shutdown the xd-admin and xd-container servers. The newest version of Jolokia has the ability to boostrap itself inside an application context vs. requiring a java agent. I suspect using the application context approach will provide us with more flexibility (e.g. property replacement etc) but not sure.",4
XD-159,"Parameter parsing does not work if an argument contains '--'.","Parameter parsing does not work if an argument contains '--'. For example: {code} ... | transform --expression=42 | transform --expression=--payload |... {code} Also, I was surprised that this worked.. {code} | transform --expression=new StringBuilder(payload).reverse() | {code} ... but this didn't... {code} | transform --expression='new StringBuilder(payload).reverse()' | {code} I think we need to tokenize the argument (with ' if contains spaces) and remove any surrounding '...' from the result. This means if someone wants a SpEL literal they would have to use something like {code}--expression=''Hello, world!''{code} resulting in a SpEL literal 'Hello, world!'",1
XD-160,"Publish golo themed docs documentation to static.springsource.org as part of nightly build","The wiki repo contains a script, gen-docs.sh, that we are planning to use to generate a pretty HTML version of the Getting Started guide. We should consider using maven (or gradle, but there is currently an issue documented in build.gradle) to generate this and other reference docs and publish them automatically as part of a nightly build.",3
XD-161,"Add HTTP Delete Stream Operation",NULL,5
XD-162,"Create design document for implementation strategy to support message conversion in ChannelRegistry","The conversion should be based on content-type headers, similar to the way Spring's HttpMessageConverters work (with mime types). Also, the map of available converters should be extensible while including the most common defaults (for JSON, XML, etc). We most likely want to add a few of our own content types also (e.g. for Tuples). Most likely, this logic and the configuration methods for extending the converter map, belong in AbstractChannelRegistry since it should be common across all implementations (i.e. the logic should be the same regardless of the transport used after-serialization/before-deserialization).",5
XD-163,"Enable grouping of modules for co-located deployment","example: {code} a | (b | c) | d {code} ...where b and c modules are deployed together as a composite module. There are 2 options (maybe more) for how we could handle that. One would be defining a CompositeModule type that simply bridges the channels (b's output to c's input in this example). The second option would be to deploy those together on the same node as modules but using the LocalChannelRegistry between them. ",12
XD-164,"Validate processing modules declare the required channels","Validate that modules have required channels declared according to their type. Currently the stream deployer accepts processors with no input, but the stream doesn't complete. We should fail earlier and more loudly.",2
XD-165,"xd-container and xd-admin should log to a file out of the box","We should have an externally editable log4j config file in a conf dir and the default should log to a file (presumably in a logs dir)",2
XD-166,"Create config support based on channel registry type","We need to have the XD container & admin reading the registry specific property based on the registry type selected. From Mark F, on one of the code review comments: Maybe rather than having redis, rabbit, etc. properties all within a container.properties we should rely upon naming conventions instead. Specifically, we could have a single configurable property for the type of channel registry (""redis"", ""rabbit"", or ""local"" being possible values), and then we could use something like: ",3
XD-167,"Script to generate reference documentation from wiki and include in .zip distribution",NULL,5
XD-168,"Decide on location to host http reference documentation and automate upload in build scripts",NULL,4
XD-169,"XD should run offline","Trying to run XD offline results in an error in redis.xml because the cloudfoundry schema file is missing. We need to add the cf-runtime jar to the classpath to resolve this.",2
XD-170,"Home wiki page improvements","Add more structure, more easily find the reference guide. The style that is here https://github.com/snowplow/snowplow/wiki is nice. ",2
XD-171,"Create project home page for SpringXD on springsource.org/spring-xd","A minimal project page of a 'top level project' page that has basic information of docs and links to the github wiki page. No need to list maven coordinates.",4
XD-172,"Create links to SpringXD on other pages of springsource.org site","bottom home page - list of projects data/integration category landing pages - related projects. ",4
XD-173,"Document how to create a custom input/output module for existing SI channel adapters","Document how to take an existing input/output channel adapters in spring integration and add them as a XD source/sink module. Should be as end-user focused, step by step guide as possible. Consider including a getting started gradle/pom.xml ",6
XD-174,"Document how to create a custom processor module.","The use case is to write custom code that does processing on a specific domain class (perhaps from twitter adapter) or a tuple. Need to package up this code so that it can be used inside XD. ",4
XD-175,"Create and document a syslog aggregation example","2-3 containers (separate processes) that the stream: syslog | tcp 1 container (separate process) that aggregates the data sent from those conainers, tcp | severityFilter | hdfs ",8
XD-176,"Support exponential moving average in RichGauge ","This could easily be supported in the existing gauge by adding a setAlpha method to RichGaugeService and adding the extra parameter ""alpha"" to the gauge data (https://en.wikipedia.org/wiki/Exponential_moving_average). If not set it would default to the current behaviour (simple mean), otherwise it would calculate the exponential moving average in place of the mean.",2
XD-177,"Easily switch between a single process that performs all admin and processing tasks to one that has a dedicated admin processes and distributed processing containers.",NULL,13
XD-178,"DefaultContainer should have a default constructor that generates a UUID","The current incrementAndGet approach based off redis will not easily be applicable in local model deployment",1
XD-179,"Have three startup scripts, xd-singlenode, xd-admin, and xd-container","The xd-singlenode script will launch a main application that creates both the admin node (to process http admin requests) and the container node (to execute modules for data processing) within in the same process the xd-admin script will launch a main application that creates only the admin node (remove current embeddedContainer options) the xd-container script will launch a main application that creates only the container node (as it is now)",1
XD-180,"The command line for xd-admin and xd-container to support an additional option, pipeProtocol, that is used to determine the middleware for sending admin requests and data between processing steps","The name 'pipeProtocol' is tentative. 1. The command line scripts for xd-admin and xd-container would support a --pipeProtocol option, with the default being to use Redis. (Otherwise use xd-singlenode). 2. The xd-admin and xd-container scripts will use the value of pipeProtocol to set the java system property xd.pipeProtocol when launching the app. ",1
XD-181,"Update launcher.xml to have protocol independent beans defined and an import statement to load protocol specific defintiions from a system property defined location.","launcher.xml can make use of the system property xd.pipeProtocol inside an import statement. This determines which version of the XD infrastructure to load, for example what ChannelRegistry implementation, Local or Redis based, or specific message listener containers. File name conventions should be used, so if the option passed in from the command line is --pipeProtocol localChannel then the XML filename looked for has the 'Protocol' suffix applied, e.g. localChannelProtocol, and is loaded via the classpath. Redis and Local will not be the only options, other implementations will be provided in the future, e.g. Rabbit, and the user may be able to provide their own implementations of these infrastructure classes (an advanced task). ",3
XD-182,"Create redisProtocol.xml that will load all the Redis specific implementations to suppor the XD container runtime and administration","The redis specific beans that are defined in the current launcher.xml should move into this configuration file. ",3
XD-183,"Create localChannelProtocol.xml that will load all the SI specific implementations to suppor the XD container runtime and administration",NULL,3
XD-184,"Add unregistration support to the channel registry",NULL,4
XD-185,"Refactor StreamServer to an interface and create Redis and Local implementations","The current StreamServer depends on RedisStreamDeployer. Call this RedisStreamServer and extract interface to allow alternate implementations",2
XD-186,"Create a pipe protocol independent StreamDeployer","Create StreamDeployer that does not depend on an adapter implementation",2
XD-187,"Create XD script for xd-single node","This script will launch XD admin along with the module container. As part of this implementation, we will also remove the embedded options for XD admin & container scripts.",2
XD-188,"Add directory to classpath in server startup scripts so groovy based processors can be easily referenced by name without a resource uri prefix","a stream such as time | filter --script=oddMinuteFilter.groovy | file would load the groovy script 'oddMinuteFilter.groovy' that is located in the directory modules/processor or perhaps in modules/processor/scripts. Not sure the benefit of having a subdirectory below processor just for scripts.",1
XD-189,"Investigate running XD on Cloud Foundry",NULL,5
XD-190,"Cleanup embedded container story","The --embeddedX options are a bit confusing in code right now, as the Admin can embed the Container and vice-versa. I guess we should only keep the Admin>Container side of things.",1
XD-191,"Move Redis connection metadata logging into the code closest to establishing that connection","XD-106 included detailed logging about the Redis metadata within the RedisContainerListener, but it seems as though that info could be logged somewhere closer to the establishment of a Redis connection for the XD runtime (and could be logged even if this listener, whose main role is to capture Container-related events, is not enabled).",2
XD-192,"Update getting started documentation to use xd-singlenode start script.","With the new option of starting without requiring redis, the getting started documentation should reflect this easier way to start processing data.",2
XD-193,"Need more unique resource locations for XD internal configuration","Currently internal config files are in META-INF/spring with fairly generic names. To avoid potential collisions if users add their own configuration in the classpath, we should have a more unique location, e.g. META-INF/spring/xd",3
XD-194,"Users should be able to package custom modules into a single jar","If a user needs to deploy a module containing custom code, they have to build a jar for the lib dir and somehow deploy the app context file separately to a modules dir. This is a bit inconvenient to build from a project, since context files in src/main/resources typically get built into a jar. Might be better to accept both jar and xml files in the modules dir, though that brings up the issue of classpath isolation.",5
XD-195,"The {{time}} Source Should Emit String by Default","When running in local mode (no Redis) {{time | tcp}} no longer works. Change the {{time}} source to emit the date as a String, while allowing an option to emit a {{Date}} object.",1
XD-196,"replace the hacky parser with a good one","Replace the existing DSL parser that uses string indexing with a more robust one based on a derivative of SpEL. This will provide a stable base on which to quickly iterate on syntax. ",4
XD-197,"Create a scriptProcessor module that allows the execution of a groovy (potentially jruby,jython) based SI Service Activator","This will enable arbitrary processing logic to be used in a processing step. See http://blog.springsource.org/2011/12/08/spring-integration-scripting-support-part-1/ would be the essence of the module. Probably 'lang' gets detected from the file extension.",4
XD-198,"Documentation for developing streams in the IDE needs to mention including scripts dir to project classpath","{{curl -X POST -d ""time --interval=3 | transform | log"" http://localhost:8080/streams/test}} results in the following stack trace in the DEBUG log. It's apparently benign, but ugly... {code} 2013-06-06 10:43:36,875 [task-scheduler-1] DEBUG: org.springframework.scripting.support.ResourceScriptSource - class path resource [transform.groovy] could not be resolved in the file system - current timestamp not available for script modification check java.io.FileNotFoundException: class path resource [transform.groovy] cannot be resolved to URL because it does not exist org.springframework.core.io.ClassPathResource.getURL(ClassPathResource.java:177) org.springframework.core.io.AbstractFileResolvingResource.lastModified(AbstractFileResolvingResource.java:170) org.springframework.scripting.support.ResourceScriptSource.retrieveLastModifiedTime(ResourceScriptSource.java:101) org.springframework.scripting.support.ResourceScriptSource.getScriptAsString(ResourceScriptSource.java:79) org.springframework.integration.scripting.RefreshableResourceScriptSource.(RefreshableResourceScriptSource.java:46) sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) java.lang.reflect.Constructor.newInstance(Constructor.java:513) org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147) org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:121) org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:280) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1035) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:939) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:271) org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:126) org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:616) org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:148) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1035) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:939) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:271) org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:126) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1118) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:589) org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925) org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:472) org.springframework.xd.module.SimpleModule.start(SimpleModule.java:97) org.springframework.xd.dirt.module.ModuleDeployer.deployModule(ModuleDeployer.java:120) org.springframework.xd.dirt.module.ModuleDeployer.handleMessageInternal(ModuleDeployer.java:108) org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:69) org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:84) org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:57) org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:102) org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:102) org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:126) org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:230) org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:129) org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:73) org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:67) org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:137) org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73) org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:115) org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:102) org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77) org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:178) org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:149) org.springframework.integration.core.MessagingTemplate.doSend(MessagingTemplate.java:304) org.springframework.integration.core.MessagingTemplate.send(MessagingTemplate.java:165) org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:92) org.springframework.integration.x.redis.RedisQueueInboundChannelAdapter.access$4(RedisQueueInboundChannelAdapter.java:1) org.springframework.integration.x.redis.RedisQueueInboundChannelAdapter$ListenerTask.run(RedisQueueInboundChannelAdapter.java:110) org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) java.util.concurrent.FutureTask.run(FutureTask.java:138) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) java.lang.Thread.run(Thread.java:662) {code} ",1
XD-199,"Ensure the DELETE Operation can Delete a Tap","While deleting a stream doesn't remove any taps right now, we should be able to explicitly delete a tap. Determine whether the current DELETE works and, if not, make it so.",3
XD-200,"Creating a tap throws an exception","Creating a tap throws an exception. In local mode: Cannot resolve reference to bean 'redisConnectionFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'redisConnectionFactory' is defined But also fails when using redis. ",4
XD-201,"Fix XD scripts on windows","Currently the XD scripts are broken in windows. ",2
XD-202,"Update twittersearch module for Twitter 1.0 API retirement",NULL,3
XD-203,"Provided modules should be integration tested","I don't see that we have automated tests for the modules we provide out-of-the-box. We could make the modules folder an Eclipse project (which would also help solve XD-198) and add some integration tests similar to those documented here: https://github.com/SpringSource/spring-xd/wiki/Creating-Custom-Modules",5
XD-204,"Document processor modules","Fill in https://github.com/SpringSource/spring-xd/wiki/Processors",2
XD-205,"Processor modules should support scripts in languages other than groovy","Filter, Transform, and Script modules all assume the provided script is written in Groovy. This is partly due to the fact that the ""lang"" attribute of can't be set to a property value (i.e. lang=""${lang:groovy}""), which would allow users to pass in the expected language. Or perhaps we could use a SPEL expression or script to pick the language based on the file extension?",3
XD-206,"XD AdminMain & ContainerMain should check xd.home property from scripts","Currently, the system property xd.home is set as JVM_OPTS (via SPRING_XD_ADMIN_OPTS) into xd-admin & xd-container scripts. Inside the ContainerMain & AdminMain, we need to check if this system property is set and use it. It seems like, this check is missing now.",1
XD-207,"Provide configurable properties for hdfs sink.","The config file modules/sink/hdfs.xml has a hardcoded value to locate the namenode. fs.default.name=hdfs://localhost:9000 the fs.default.name proprety should be configurable and we should also support loading an external configuration file using ",2
XD-208,"Document the file sink",NULL,1
XD-209,"Create or document existing project template for custom module creation","The wiki docs on creating custom modules have entire build.gradle files in them. Would be good to explore existing STS templates, maven archetypes, etc or create new ones for authoring custom modules.",3
XD-210,"If output directory does not exist for a file sink, by default allow it to be created","There shouldn't be a need to do a mkdir -p before sending data to a file sink.",1
XD-211,"Document the log sink",NULL,1
XD-212,"Add http port command line option to AdminMain","Currently StreamServer has setPort, but no way for end user to set it. ",2
XD-213,"Temporarily add toString() Logic in Local Mode Inter-Module Comms",NULL,2
XD-214,"Create documentation on the general DSL syntax","The asciidoc wiki should have a section (included in the _Sidebar.asciidoc as well) that describes the general usage of the DSL syntax.",3
XD-215,"Add authentication information to twittersearch source doc","Since the changes for XD-202, twittersearch requires authentication. Need to update the docs to reflect this.",1
XD-216,"Add support for tap foo.bar syntax in the DSL",NULL,2
XD-217,"Add config dir to classpath to support custom properties-locations","The transform, filter, and script processor modules support passing in a properties-location for script variables. We need a default location on the classpath for users to provide custom properties files.",2
XD-218,"Add support to load a twitter.properties file in the source",NULL,1
XD-219,"Surpress tap WARNING message in local mode",NULL,1
XD-220,"Add twitter oauth properties file to config dir","Those property keys should then be provided as defaults for the placeholders in source/twittersearch.xml",2
XD-221,"Links in asciidoctor generated HTML+docbook documentation are broken","The issue arises because the link:document[Label] asciidoc macro is meant for ""external documents"" and creates {{}} in docbook / {{}} in html, whereas we want {{}} / {{}} resp. We also want it to continue working in github live view. I guess what could work is to have the macro (either override the link macro or create our own if github supports that) that looks like : {{link:document#anchor[Label]}} (the #anchor works out of the box in asciidoc and should work in github) but override it for the html and docbook backends to render to the correct form. The thing is, there are several ways to create/override macros (and templates they render to), some of which make sense to our setup: - having asciidoc.conf in the directory of the processed document (http://asciidoc.org/userguide.html#X27) - having docbook.conf/html.conf in the directory of the processed document (http://asciidoc.org/userguide.html#X27) - defining macros using attributes (http://asciidoc.org/userguide.html#_setting_configuration_entries) I tried all of those, but to no avail. These DO WORK with plain asciidoc, but not with our toolchain. Don't know if the problem is with asciidocTOR or with the gradle wrapper though. ",2
XD-222,"Add docs for Deleting a simple stream.","curl -X DELETE http://localhost:8080/streams/ticktock",1
XD-223,"Update Creating a Custom Source Module doc with a different SI adapter due to Twitter issues","https://github.com/SpringSource/spring-xd/wiki/Creating-a-Source-Module uses the SI twittersearch inbound channel adapter, which is no longer going to work once Twitter disallows anonymous searches. Ideally we update the example to use a new version of SI-twitter that adds support for this (as opposed to the XD workaround.)",2
XD-224,"Reduce necessity for quoting in parameter values in DSL expressions","parameter values that include spaces need to be quoted, but this becomes overly complex in this kind of case. Here is what you want to say: {code} http --port=9995 | filter --expression=payload.matches('hello world') {code} With the rule 'parameter values that contain spaces must be quoted' it would be this: {code} http --port=9995 | filter --expression='payload.matches('hello world')' {code} But then to include single quotes within a single quoted string you need to use two of them: '' - so it becomes {code} http --port=9995 | filter --expression='payload.matches(''hello world'')' {code} Less than ideal. ",2
XD-225,"field-value-counter should support nested fieldNames","for example, when using the 'twittersearch' source module, the ""hashTags"" are nested within ""entities"", and the value for hashTags is itself an object with a ""text"" field, so the following would be needed to count the actual value of interest: {code} tap @ tweets | field-value-counter --fieldName=entities.hashTags.text {code} ",2
XD-226,"Cleanup and Optimize gradle tasks to bundle spring-xd distribution","We need to cleanup some of the duplicate gradle tasks that bundle spring-xd distributions. Currently, distXD does the copy of distributions from ""spring-xd-dirt"", ""redis"" and ""spring-xd-gemfire-server"" projects into ""$rootDir/dist/spring-xd"". And, the task ""zipXD"" makes the zip archive. These tasks should be combined with the ""distZip"" & ""docZip"" tasks. We also need to remove the duplicate artifacts configuration from these tasks.",2
XD-227,"Add jetty-util-6.1.26.jar and jsr311-api-1.1.1.jar as required jars so they will be on the XD classpath","This is needed for the use of the webhdfs:// scheme to talk to HDFS over http.",1
XD-228,"Missing '=' in example of http stream","In documentation attached to M1, in Streams/Introduction section, there's {noformat} http --port 8091 | file --dir=/tmp/httpdata/ {noformat} while it should be: {noformat} http --port=8091 | file --dir=/tmp/httpdata/ {noformat} missing ""{{=}}"" in {{http}}",1
XD-229,"Add RabbitMQ-based implementation of ChannelRegistry",NULL,8
XD-230,"Add RabbitMQ source module","configurable parameters should include the queue-name(s) and optional binding key pattern connection info, such as host and port, should also be configurable but with defaults (localhost and default port), and that should likely fallback to a rabbit.properties file in the $XD_HOME/config directory",3
XD-231,"Add Twitter gardenhose source module","we have a prototype gardenhose adapter that was built directly upon RestTemplate (streaming on a background thread), but Spring Social Twitter has an issue on its 1.1 roadmap that is relevant: https://jira.springsource.org/browse/SOCIALTW-2 ",3
XD-232,"A HATEOAS designed REST API using Spring HATEOAS library.",NULL,21
XD-233,"Create an shell to control all aspect of stream/job management",NULL,21
XD-234,"Support various output format, e.g. Avro, SequenceFile, more advanced rollover options.",NULL,21
XD-235,"Modules (sinks, processors, sources) should be able to be easily tested inside the IDE using JUnit ",NULL,21
XD-236,"Create an Aggregate Counter","An aggregate counter rolls up counts into discrete time buckets. There is an existing POC implementation in Java based off the library https://github.com/thheller/timed-counter The README there has a good description of the desired feature set.",8
XD-237,"Spring Batch jobs should be able to be deployed to the DIRT runtime and managed.",NULL,21
XD-238,"Deploying Custom Code","When a module is deployed, it should run in its own isolated classpath. The current code has all dependencies in a single classpath, taken from the lib directory at startup. This has a number of drawbacks, one of the most important is the batch jobs can not be contributed to the system at runtime. The work for this epic is decoupled from any module deployment story. The assumption is that there will be a directory layout as shown below. Current layout ./modules/. |-- common |-- job |-- processor |-- sink |-- source |-- trigger And inside source |-- source | |-- file.xml | |-- gemfire-cq.xml | |-- gemfire.xml | |-- http.xml | |-- jms.xml | |-- mqtt.xml | |-- rabbit.xml | |-- syslog-tcp.xml | |-- syslog-udp.xml | |-- tail.xml | |-- tap.xml | |-- tcp.xml | |-- time.xml | |-- twittersearch.xml | |-- twitterstream.xml Using an example of the source directory from the current layout.e.g ./modules/source/file, the new layout would be ./modules/source/file/lib/spring-integration-file.jar ./modules/source/file/config/file.xml We should support both the new and old layout styles simultaneously. There what is under 'file' directory is the 'package' No .zip, war, is required.",21
XD-240,"Throughput optimized TCP based adapters based on Reactor TCP project",NULL,21
XD-242,"Add JMS source module",NULL,2
XD-244,"Create a Trigger","h2. Narrative As the XD system, I need to be able to execute a job (or potentially a stream) based on a given condition (time, data existence, etc). This story is intended is for a local trigger implementation but remote triggers will also need to exist. h2. Acceptance Criteria # Implement the ability to register a time based trigger {{trigger }} for example # Implement the ability to register a file existence based trigger {{trigger }} for example # Implement the ability to execute a job via an anonymous trigger: {{job_name @ }} # Implement the ability to execute a job via a job via the previously registered trigger: {{job_name @ trigger_name}} ",8
XD-245,"Deploy Batch Jobs on XD","h2. Narrative As a developer, I need a way to deploy job configurations as well as the related custom code to XD. h2. Acceptance Criteria # Provide the ability to register jobs that have been deployed as modules via something like {{curl -d ""job"" http://localhost:8080/streams/myJob}} where job is the name of the job definition located in /modules/job and myJob is the name of the resulting registered job # Confirm that both ""regular"" jobs and Spring Hadoop based jobs can be packaged/run.",8
XD-246,MessageChannelItemWriter,"h2. Narrative As a user of XD, I want to be able to use a job as a source. To do so, we need the output of a job to be written to a message channel h2. Acceptance Criteria # Create a new ItemWriter in the Spring Batch project to write to a Spring Integration message channel.",4
XD-247,"Need to be able to specify password for Redis","Running on Cloud Foundry (and other managed environments) we need to be able to specify a Redis password in addition to host and port.",2
XD-250,"Set up a project for XD REST client library",NULL,5
XD-251,"Add support for stream creation","For stream creation we need to be able to specify: source sink processor - filter - transformer - script etc. ",10
XD-255,"Set up a project for XD Shell","Set up a basic Spring Shell project for XD Shell",3
XD-256,"Create a banner page for XD Shell",NULL,2
XD-257,"Create the base implementation for XDCommands for the shell","This is the basic setup of the commands file - no specific command implementations",3
XD-258,"Add command for stream creation",NULL,3
XD-259,"Add command for tap creation",NULL,1
XD-260,"Add command for listing streams",NULL,3
XD-261,"Add command for deleting a stream",NULL,1
XD-262,"Convert current REST servlet to Spring MVC",NULL,3
XD-263,"Support GET /streams","Pagination support, maybe querying by name as well",5
XD-264,"Spring MVC infrastructure tests",NULL,8
XD-265,"Retrieve description of all registered modules ","GET /streams/{streamname}/modules and GET /streams/{streamname}/modules/{modulename} The former returning links to the latter ",5
XD-266,"Create java client lib over REST API","So that clients (e.g. Shell or custom user program) are insulated from REST details (ala Cloud Foundry). May go even further if we want a Java DSL for stream definitions (that may reuse Batch command POJOs btw): Difference between: xdClient.createStream(""mystream"", ""http --port=9000 | file"") and import static stuff.*; StreamDef stream = http().port(9000).pipe(file()); xdClient.createStream(""mystream"", stream);",8
XD-267,"Job as a Source","h2. Narrative As an XD developer, I need to be able to use a batch job to stream data as a source. h2. Acceptance Criteria # Implement the ability for a job to be defined as a source in the DSL # Add the configurations for the batch infrastructure transparently to the user # Add the ability to specify if the job is stateful (picks up where it left off if it stops or restarts at the beginning).",10
XD-268,"Streamline command-line arg management","Command line arguments (and especially their default values) are currently scattered around different places. The aim is to regroup those in a common place (*Options classes make sense). Also, not very happy with how System properties are used as a vehicle for options.transport / options.home",2
XD-269,"Create JobDefinition repository","h2. Narrative As XD, I need a persistent way to register job definitions (beyond the map registry implementation provided by Spring Batch). h2. Acceptance Criteria # XD should be able to register, unregister, and find job definitions via the registry. # The registry should be backed by Redis so that it is persistent.",4
XD-270,"The HDFS Sink should support a file naming strategy to distinguish between file currently being written and completed files","A file that is in the process of being written to should have a customized suffix added to the name, e.g. 'temp'. Once the file is closed, the suffix is removed and replaced with another value - default value can be dependent on the serialization format used, but can be customized",8
XD-271,"The HDFS Sink should support a number of rollover options","A strategy to roll over files that allows the user to choose between 1) the size of the file 2) the number of events/items in the file 3) an idle timeout value that if exceeded, will close the file",8
XD-282,"The HDFS Sink should support writing POJOs to HDFS using Avro Serialization","Writing POJOs using CDK Data (Avro) We should support both partitioned and un-partitioned. This story addresses only un-partitioned. Document limitations in terms of which Java types are supported and not supported by the Avro serialization ",10
XD-290,"Redis backed container's RedisQueueInboundChannelAdapter is not performant","Currently, the RedisQueueInboundChannelAdapter has blocking operation when pulling the messages out of redis queue and this is not performant. There are few ideas from the discussion to make it better: 1) Get more items from the redis queue per connection 2) We will also have compression of messages(at the channel registry) before being sent to the redis queue We also need to investigate what redis connection strategy makes the RedisQueueInboundAdapter better. ",2
XD-291,"HTTP Source still listens on port 9000 after removal.","Steps to reproduce: 1. curl -d ""http | log"" http://localhost:8080/streams/testHttp 2. curl -X DELETE http://localhost:8080/streams/testHttp 3. curl -d ""http | log"" http://localhost:8080/streams/testHttp org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:9000",1
XD-292,"Redis 'install-redis' script fails on Ubuntu64","The installation script for redis fails on Ubuntu64 when trying to untar the redis distribution. The script uses REDIS_ZIPNAME instead of REDIS_ZIP_PATH. This bug will be seen on any Linux 64 bits platform and looking at the code, even Linux 32 bits platform.",1
XD-293,"Ensure package-info.java is present for each package","We should ensure that each Package of Spring XD is documented. Right now the created JavaDoc looks barren: http://static.springsource.org/spring-xd/docs/1.0.0.M1/api/ ",4
XD-294,"Apply JavaDoc HotFix",NULL,1
XD-295,"redis.properties values ignored","The container application loads {{redis.properties}}, but for some reason the values are ignored, and defaults are used instead. Repro steps: # Unpack Spring XD 1.0.0.M1 to a machine with no running Redis instance # Change /xd/config/redis.properties to specify a different hostname # Run /xd/bin/xd-container # Observe error about inability to connect to Redis on localhost Workaround * Pass -Dredis.hostname={desired IP} as a JVM parameter",1
XD-296,"Add log config file to gemfire in final distro","The changes for XD-144 mean that log4j files are no longer in the library jars. The admin server already has a logging configuration which should be activated by the startup scripts, but the separate gemfire app doesn't.",1
XD-297,"File sink should support rollover","I wanted to have a rollover feature when I was streaming tweets to a file overnight, just to avoid dealing with a single enormous file (in case I collected more data than my demo could handle and needed to split it up).",2
XD-298,"Refactor gardenhose into more generic twitterstream source","Twitter's streaming APIs have more capabilities than just the plain statuses/sample.json. In particular we should support the filter.json option and the use of ""track"" (https://dev.twitter.com/docs/streaming-apis/parameters#track) as well as other request parameters (delimited, language etc).",2
XD-299,"Creating a tap with same name as existing streams results in infinite loop","See http://stackoverflow.com/questions/17157068/counter-analytics-in-springxd The underlying issue is stream creation with a name already taken though",5
XD-301,"Request to create a repo for Spring XD performance testing","It would be nice if we have a git repo for Spring XD performance testing. This would enable us to have a common repository (rather than inside spring-xd as a subproject) for all performance related code specific to any module, message middleware etc., ",4
XD-302,"User wants ability to create a mock source","To send a pre-set message to process(es)",8
XD-303,"User wants ability to create a in-process sink or tap","So that we can validate the message content in the stream",8
XD-304,"User wants ability to test processors","Be able to point to the processor xml file, e.g. modules/processors/transformer.xml, and have access to a source channel that drives messages into the processor and a output channel where output messages are send. The outbound channel is queue backed. Test sending JSON to a processor module that uses Tuples. ",8
XD-305,"User wants ability to test sinks","Handled by 1245",8
XD-306,"User wants ability to test sources","Examples: 1. Be able to start the rabbitmq source just by pointing to modules/source/rabbit.xml, pass in some property file for parameters to be replaced, and outgoing message is placed in a in-memory queue backed channel for use with assertions to verify functionality. 2. Test for as many source types as is 'reasonable', e.g. MQTT/TCP testing might be harder than say rabbitmq. 3. Test that sending json, results in media-type header is set to json 4. Test that sending POJO, "" POJO 5. Test that sending Tuple, "" Tuple 6. Test that sending raw bytes, "" raw bytes ",8
XD-307,"User to send a message directly to module and receive a message from a module",NULL,8
XD-308,"User wants ability to test multiple processors in a chain",NULL,8
XD-309,"Users should be able to package custom modules as a single zip file","See XD-194 for additional considerations. Zip support should be similar to uber-jar, or possibly replace uber-jar support.",8
XD-312,"Add Jolokia Agent Depending on Run Mode","WAR Vs. JVM Jolokia Agent Jolokia Vs. JVM MBeanServer Probably needs support for Spring Profiles.",5
XD-313,"Add Spring/Integration MBean Exporters to Module ApplicationContexts","Global option? Override for individual modules? module types?",8
XD-315,"Package Shell ""binary"" next to xd-admin and xd-container","The shell should be an 'executable' delivered out of the box in much the same way that xd-container and xd-admin are right now. If we follow how redis/mongo distribut the shell, it sits side by side with the other binaries",3
XD-316,"Create a common exception framework for XD","Need to capture exceptions from the various projects that make up XD and wrap them in XD Specific exceptions. An example of this is when leaving out the channels in the module definitions, we see NoSuchBeanExceptions and IllegalArgumentExceptions thrown based on which module and what channel is missing. ",5
XD-317,"Add Documentation Chapter on Executing Batch Jobs",NULL,4
XD-318,"Create Splunk sink module","This would be based off the spring-integration-extenstions splunk project. The use of this adapter for storing tweet data is in https://github.com/markpollack/springone We should be able to reproduce the use case as done in that demo",10
XD-319,"Add command for deleting a tap",NULL,1
XD-320,"Support for GET of /taps",NULL,1
XD-321,"Add create() and deploy() to TapsController",POST?,1
XD-322,"Support for DELETE of taps",NULL,1
XD-323,"Add command for listing of taps",NULL,1
XD-328,"Retrieve information for a Counter",NULL,5
XD-329,"Retrieve information for a Field Value Counter","TODO as part of this (see XD-537): * Get rid of so-called Service layer in analytics project (doesn't do much right now, and logic would better live in the 'Handler' IMO) * Have REST controllers depend on XRepository in all cases",3
XD-330,"Retrieve information for a Gauge","TODO as part of this (see XD-537): * Get rid of so-called Service layer in analytics project (doesn't do much right now, and logic would better live in the 'Handler' IMO) * Have REST controllers depend on XRepository in all cases",3
XD-331,"Retrieve information for a Rich Gauge","TODO as part of this (see XD-537): * Get rid of so-called Service layer in analytics project (doesn't do much right now, and logic would better live in the 'Handler' IMO) * Have REST controllers depend on XRepository in all cases",3
XD-332,"Retrieve information for an aggregate counter","TODO as part of this (see XD-537): * Get rid of so-called Service layer in analytics project (doesn't do much right now, and logic would better live in the 'Handler' IMO) * Have REST controllers depend on XRepository in all cases",3
XD-334,"Document the structure of the REST API",NULL,5
XD-335,"Review DSL","updated story points to 14 since 5 of us just participated in a 2 hour call, and we still need to discuss ""topology"" support after some dev spikes later this week",14
XD-336,"Document Splunk source sink",NULL,2
XD-337,"Test connection pooling on Redis blocking/nonblocking operations",NULL,2
XD-338,"Create tcp/udp load generator script for XD performance testing","Create a load generator script which can generate messages at specific 1) Rate 2) Payload 3) Concurrency to a specific tcp/udp port where a syslog adapter is listening.",4
XD-339,"Investigate using profiler when doing the performance testing","Investigate how efficiently we can integrate profiler into the performance test.",1
XD-340,"Create script to extract table data from JSON based on a given HAWQ table structure","We should be able to write a script that can examine the table structure for a given HAWQ table and then extract the data from JSON without the custom script we are using now.",8
XD-341,"Document JMX features","Document jmx command line options and refer to jolokia",2
XD-342,"Fix classpath error caused by multiple conflicting servlet-api jars","There is some conflicting Servlet API jars on the claspath that needs cleanup. Building and running with xd-singlenode script gave this error: Jun 27, 2013 3:18:16 PM org.apache.coyote.http11.AbstractHttp11Processor process SEVERE: Error processing request java.lang.NoSuchMethodError: javax.servlet.ServletContext.getEffectiveSessionTrackingModes()Ljava/util/Set; org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:674) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:402) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) java.lang.Thread.run(Thread.java:680) ",3
XD-343,"Investigate JMX object naming of deployed modules and inbound/outbound channel adapters.","The object naming is still not ideal for XD since SI conventions add some noise. Likely need to design and implement a custom naming strategy",5
XD-344,"Add BatchMbeanExporter for batch modules",NULL,5
XD-345,"Replace ""gardenhose"" doc with new ""twitterstream""",NULL,1
XD-346,"Esper based Complex Event Processing module","I would like to see a module created that supports complex event processing. I have reviewed GemFire Continuous Query but was not able to find a feature for time windows. I have used Esper in the past for this type of processing.",1
XD-347,"Investigate Redis connection timeout issues when running performance test","With the performance test run, the numbers (messages sent/received per second) keep varying as there are ""redis client connection timeout exceptions"" (Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out) at both redis inbound/outbound channel adapters as I increase the total number of messages being processed (max. 10K/second). Some of the exception messages for the review: 1) With connection pool (at Redis outbound): Caused by: org.springframework.data.redis.connection.PoolException: Could not get a resource from the pool; nested exception is com.lambdaworks.redis.RedisException: Unable to connect at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:95) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:36) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createLettuceConnector(LettuceConnectionFactory.java:318) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:109) at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:81) at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:53) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:157) at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:137) at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:84) at org.springframework.data.redis.core.DefaultListOperations.leftPush(DefaultListOperations.java:71) at org.springframework.data.redis.core.DefaultBoundListOperations.leftPush(DefaultBoundListOperations.java:67) at org.springframework.xd.perftest.redis.outbound.RedisQOutboundChannelAdapter.handleMessageInternal(RedisQOutboundChannelAdapter.java:71) at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73) ... 17 more Caused by: com.lambdaworks.redis.RedisException: Unable to connect at com.lambdaworks.redis.RedisClient.connect(RedisClient.java:176) at com.lambdaworks.redis.RedisClient.connectAsync(RedisClient.java:139) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool$LettuceFactory.makeObject(DefaultLettucePool.java:252) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1181) at org.springframework.data.redis.connection.lettuce.DefaultLettucePool.getResource(DefaultLettucePool.java:93) ... 29 more Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:6379 at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) 2) Without connection pool (at Redis inbound): Caused by: com.lambdaworks.redis.RedisException: Unable to connect at com.lambdaworks.redis.RedisClient.connect(RedisClient.java:176) at com.lambdaworks.redis.RedisClient.connectAsync(RedisClient.java:139) at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.createLettuceConnector(LettuceConnectionFactory.java:321) ... 12 more Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: localhost/127.0.0.1:6379 at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ... 3 more",2
XD-348,"Trigger - Add support for fixed-delay interval","Trigger - Add support for fixed-delay interval",1
XD-349,"Trigger - Add support for date-based one-time execution","Currently Jobs can be either executed using cron expression or immediately at once. We should also support the one-time scheduling of jobs in the future. Would this possibly require us to implement schedule-persistence? That could severely impact story-points.",4
XD-350,"Support hourly resolution in redis aggregate counter",NULL,4
XD-351,"Support daily query resolution in redis aggregate counter",NULL,4
XD-352,"In-memory implementation of aggregate counter",NULL,3
XD-353,"Clean up Spring Configuration","- Container context should be separate from Admin Context in local mode (consistency across transports). Provide a LocalChannelRegistry to bridge deploy and undeploy channels to the ModuleDeployer - Verify Plugins are not in common module context. They are only needed by the ModuleDeployer not the Modules. - Add global-beans Config for beans to be shared among admin and container (and available to modules). This would be set as the parent context where needed. (Currently ‘analytics-context.xml’ and JobRepository shared by Admin and modules) - Fix ModuleDeployer sets parentContext twice - Rename common.xml to module-common.xml NOTE: Analytics parent only required in local mode - Write unit tests to verify configurations are as expected. Correct bean implementations and no stray beans or redundant instances where not needed - Decouple Command options from System properties (in general XD property names), this gives us some flexibility in mapping to properties or profiles where appropriate ",5
XD-354,"Investigate Reactor-based Dispatchers in the common ApplicationContext that can be used by Modules",NULL,8
XD-355,"Enable configuration of Executors in source and sink modules, by default using Dispatchers in parent context","Need to understand how individual modules may or may not share Dispatchers that are part of the parent context. If modules have their own dispatchers, those also need to be configurable.",8
XD-356,"Test startup scripts on windows","startup scripts on windows should be tested, xd-admin, xd-container, xd-shell.",1
XD-357,"Creating a base class for Plugins ","It might be worth creating a base class for Plugins that combines common concerns across plugins. E.g. That would allow us to hide the commonApplicationContext and BeanDefinitionAddingPostProcessor for common cases, instead exposing a simple addBeanDefinition method to sub-classes.""",4
XD-358,"Add support for creating named cron triggers","Simple cron based triggers",2
XD-359,"Add support for creating a spring batch job that has an embedded trigger expression",NULL,1
XD-360,"Add support for creating a spring batch job that references a named trigger",NULL,2
XD-361,"Create a trigger from Shell",NULL,1
XD-363,"Add support for creating fixed delay/ fixed rate triggers",NULL,1
XD-364,"Create TriggerDefinition Repository ","Redis based.",1
XD-365,"Support having multiple property placeholders defined in different modules","Right now the PPC for jolokia mgmt is conflicting with the PPC used to resolve redis properties. Need to determine a strategy such that multiple PPCs can be used.",4
XD-366,"Create a XD job definition","when posting the DSL to create a spring batch job e.g. ""trigger job.xml --option1=foo"" it should be stored (in redis) so that a listing of XD job definitions can be retrieved.",3
XD-367,"Final review of REST API structure document for streams, taps and jobs","Get closure on open discussion points for REST API wrt to streams, taps and jobs. ",3
XD-368,"Improve connection handling in RedisAggregateCounterService.","This is currently too chatty. It should be possible to use a single connection for each ""increment"" operation.",2
XD-369,"Further DSL extensions","Extend the DSL in the following ways: - stream naming, use '=' {code} mystream = http | file {code} - module aliasing (for later referencing) use