Files
2025-02-06 20:33:26 +02:00

355 KiB

1issuekeytitledescriptionstorypoint
2USERGRID-16Asset data does not correctly obey contextual ownership like the entity"The asset data endpoint /assets/UUID/data does not correctly obey contextual ownership. For instance, if the default role permission are set to this after removing all existing. GET,PUT,POST,DELETE:/users/me/** A user should only be able to perform the operations on their entity /users/me, and all sub collections. For instance the following scenario should work as described. # App default role permissions are edited to match the path above # User ""bob"" registers for app # User ""bob"" creates the following asset and uploads data. /users/me/assets/myasset and /users/me/assets/myasset/data # User ""fred"" registers for app # User ""fred"" should get a 404 on both /users/bob/assets/myasset, and /users/bob/assets/myasset/data # Anonymous user should get a 404 on both /users/bob/assets/myasset, and /users/bob/assets/myasset/data See org.usergrid.rest.applications.users.OwnershipResourceIT for some examples. "3
3USERGRID-17Expose refresh token at the REST tierNeed to add refresh token capability to the rest tier. May be able to update our oauth impl. Also, refresh tokens were in the core but were removed some time ago. Might be able to go back to commits prior to it being removed to see it. 3
4USERGRID-19Bad geo query returns entire collection"When a badly formed geo query is sent with a GET, the API returns the entire collection, when it should return nothing, or return some sort of query parse error. For example, this is missing the 'location' param of the query statement: within 16000 of 37.774989,-122.419413 and returns the first 10 entities in the collection even though the query can't be parsed. "3
5USERGRID-21Empty string doesn't remove an entity property (null doesn't work either)"Users should be able to remove entity properties using either empty string or null 1. Test to see if either works 2. if empty string does not work, then setting empty string should keep the property but set it to an empty string 3. null should always remove the key curl -X POST -i -H ""Authorization: Bearer YWMt4i6fzEJREeOh0FGOwhrLOAAAAUIzm9d9wtwQTdewhc1I1Kg8d8yyMOVVQv0"" ""htts://api.usergrid.com/1hotrod/fdfdsafd/books/8dd42b7a-4325-11e3-a8a9-2be66bc83082?ql="" -d '{""fred"":""""}' or curl -X POST -i -H ""Authorization: Bearer YWMt4i6fzEJREeOh0FGOwhrLOAAAAUIzm9d9wtwQTdewhc1I1Kg8d8yyMOVVQv0"" ""htts://api.usergrid.com/1hotrod/fdfdsafd/books/8dd42b7a-4325-11e3-a8a9-2be66bc83082?ql="" -d '{""fred"":null}'"3
6USERGRID-23Fresh Admin user token won't work on /management/users/me"After logging on: curl -X POST ""https://api.usergrid.com/management/token"" -d '{""grant_type"":""password"",""username"":""fdsafdsa"",""password"":""fdsafdsafdsa""}' The user is not able to log in with the token: curl -X GET -i -H ""Authorization: Bearer YWMt997XKhAyEeO7oblzQPxkkAAAAUDrIyfHE8GIZlo8IhJztOX2JQ7kXpbFTHc"" ""https://api.usergrid.com/management/usersfdsafdsa"" But the token works with other endpoints. ***Update***: Looks like the problem is that the /management/users/<username or email> endpoint is not case-insensitive on the username or email address. We need to update that API call so that it is case-insensitive. "3
7USERGRID-24Collection counters on App endpoint are completely off"The counts of items in each collection returned in a call to /org/app are completely off (usually by a factor of 10). This is possibly related to USERGRID-17. Can we do something to fix this? Possible solution in two parts: 1. Investigate all code paths to ensure that counters are being invoked when needed 2. Create a job that can be invoked automatically or manually that will recount all entities in a collection "3
8USERGRID-26Offer an option to delete all inbound and/or outbound connections on entity delete"Per Ed, all inbound connections should be cleaned up when an entity is deleted: For example, if you have entity rock and entity paper who both like entity scissors, both likes connections to scissors should be deleted if scissors is deleted. It would be great if there was an option to delete all inbound connections when an entity is deleted (the same flag should also be possible on deletes by queries)."3
9USERGRID-27Limit on connection query not workingAlso "limit" doesn't seems to be working for connections for e.g. https://api.usergrid.com/fdsafdsa/test/users/me/connecting/shares?limit=2&access_token=YWMtSQ92bBCsEeOYAQ1TRlcn_AAAAUDuPjgtlh06pENBKFRmoxta8jg1XwRuwVs. Returns all in our case is 33
10USERGRID-29We don't seem to be parsing '++' correctly"I have made a simple app to test Usergrid's capabilities and I'm having trouble with updating entities that contain html entities. i.e. I add C++ as an entity in a collection called language and I also have C in the collection too. Whenever I try to update C++, which I grab via a collection query ""select * where owned='testuser' and lang='C++'"" I get back the C entity. I have tried encoding the strings , but C%2B%2B gets double encoded to C%252B%252B This a typical call it makes without me trying to encode anything: https://api.usergrid.com/clydebyrdiii... Although it seems to encode correctly it returns the entity with lang='C' and not lang='C++'; And confirmed by Scott: Nope. If I create an entity with an attribute = 'c++', I can find it by querying for: 'c*' but not for 'c' or 'c+++'. "3
11USERGRID-30/collection/id/connected/* paths should either return an entity, an empty set or a 404"Not clear what that path is supposed to do but itÕs accepted, and always returns an empty set of results or an error Examples: curl http://api.usergrid.com/fdsafdsa/sandbox/users/tim/connected/followers => returns empty set despite follower relationships existing curl http://api.usergrid.com/fdsafdsa/sandbox/users/tim/connected/users => returns empty set despite users being connected curl http://api.usergrid.com/fdsafdsa/sandbox/users/tim/connected/d2740256-e312-11e1-ad17-12313d2b9232 => returns an error despite an entity with that UUID being connected 1. Either we should accept a name or uuid at the end, and return that entity if it exists in the ÒconnectedÓ entities, or a 404 if it does not exist in the list of connected entities 2. Or we should accept a relationship identifier there and return all connected entities through that identifier, or an empty set if there are none 3. Or we should accept a collection name / type there and return all connected entities of that type, or an empty set if there are none 4. Or you should tell me what is supposed to be provided there so we can put that in our docs and SDKs :p"3
12USERGRID-35Cannot delete an entity that has a connection to it"To repro, attempt to delete an entity that has a connection to it such as: curl -X DELETE ""https://api.com/fdsafdsa/testapp/dog/Dachsund"" Notice that you get error message: {""error"":""class_cast"",""timestamp"":1386200483978,""duration"":0,""exception"":""java.lang.ClassCastException"",""error_description"":""org.usergrid.persistence.cassandra.ConnectedEntityRefImpl cannot be cast to org.usergrid.persistence.cassandra.ConnectionRefImpl""} It is expected that the entity is deleted and the connection is deleted as well."3
13USERGRID-40ActivityStreams should be indexed"top-level properties like category, content, title should be indexed by default, so you can do /activities?ql=category = 'Report' Objects such as actor, object, provider, generator & target should be deep-indexed so you can do queries like: /activities?ql=object.uuid = 'ca16c9a1-ab5b-11e1-8b99-1231381c404f' /activities?ql=author.name = 'Tim' etc."3
14USERGRID-51the result from the Setup call (to setup DB) should not say OK when it failed to create KS (or other critical failure)**Need to verify that this is still an issue "When the /setup call is run and create UG keyspaces, they could fail (for various reason). However, it seems in the result from the call, the status always says OK: {panel} [root@rut000ea lib]# curl -u superuser:supervisor 0:8080/system/database/setup \{ ""action"" : ""cassandra setup"", ""status"" : ""ok"", ""timestamp"" : 1383342928369, ""duration"" : 65 \} {panel} Here is 1 way to reproduce this problem: in the usergrid properties, define to use NetworkTopologyStrategy AND replication_factor: {panel} cassandra.keyspace.strategy.options.replication_factor=3 cassandra.keyspace.strategy.options.us-east=3 cassandra.keyspace.strategy=org.apache.cassandra.locator.NetworkTopologyStrategy {panel} In C* 1.2, it becomes stricter on the options used (previously a warning). The above combination of the strategy options is invalid for NetworkTopologyStrategy, and will result in failure of creating the keyspace. Hence the status result from the setup call should reflect it."3
15USERGRID-56Error and possible race condition in the assets/uuid/data endpoints. It looks like we might have a race condition in the assets/uuid/data endpoints. perhaps the assets in general. I noticed my calls were successful about 5% of the time. I put in a 1s wait, and now it's over 95%. When it fails, I get this error: {"error":"web_application","timestamp":1391230754494,"duration":0,"exception":"javax.ws.rs.WebApplicationException","error_description":"com.sun.jersey.api.MessageException: A message body reader for Java class com.sun.jersey.multipart.FormDataMultiPart, and Java type class com.sun.jersey.multipart.FormDataMultiPart, and MIME media type application/octet-stream was not found.\n"}3
16USERGRID-58Application end-point returns incorrect countsHere's an example of the obviously bad counts: "dogs" : { "title" : "Dogs", "count" : -248, "name" : "dogs", "type" : "dog" }, "medications" : { "title" : "Medications", "count" : 3, "name" : "medications", "type" : "medication" }, "somethings" : { "title" : "Somethings", "count" : -12, "name" : "somethings", "type" : "something" }, I suspect 3
17USERGRID-59Mismatch application metadata ( APPLICATIONS_CF and Management Apps )Sometimes, there's mismatch application metadata ( APPLICATIONS_CF and Management Apps ) So, we need to repair the datas.3
18USERGRID-82Cannot change custom entity nameIf you have a custom entity and attempt to update the entity's name (via PUT), the system will accept the request and return success (200), but the name will remain unchanged. eg. using ugc: ugc create foo "name: 'bar'" ugc update foo/bar "name: 'baz'" (Note: Referencing the entity in the PUT via UUID has the same effect.)3
19USERGRID-98POST to /events returns entity that does not existWhen I create a new event entity, the API returns an entity with uuid, but when I do a GET on the entity, the API returns a service resource not found error. GET on the /events endpoint also shows that the entity was not saved. So this POST: curl -X POST "https://api.usergrid.com/fdsafdsa/test/events" -d '{"timestamp":201111211437, "category":"testcat"}' Returns this response: { "action" : "post", "application" : "4a1edb70-d7a8-11e2-9ce3-f315e5aa568a", "params" : { }, "path" : "/events", "uri" : "http://api.usergrid.com/fdsafdsa/test/events", "entities" : [ { "uuid" : "c8a283c6-e79b-1038-8735-1d2142858146", "type" : "event", "created" : 1372888238225, "modified" : 1372888238225, "timestamp" : 201111211437, "category" : "testcat", "message" : null, "metadata" : { "path" : "/events/c8a283c6-e79b-1038-8735-1d2142858146" } } ], But a GET on /events/c8a283c6-e79b-1038-8735-1d2142858146 returns this error: {"error":"service_resource_not_found","timestamp":1372888258143,"duration":0,"exception":"org.usergrid.services.exceptions.ServiceResourceNotFoundException","error_description":"Service resource not found"} 3
20USERGRID-103Upgrade to latest Jersey 2.x releaseJAX-RS 2.0 brings lots of new features, much better extensible and async support are ones I think most useful 5
21USERGRID-160select (filter) queries with dot notation don't return resultsSteps to reproduce: 1. Create a collection and populate it with an entity with the following data: { "foo.bar": "baz" } 2. Run a GET query on this collection with a filter: https://api.usergrid.com/org/app/collection/?ql=select foo.bar 3. No results are returned because the "select" filter isn't working with dot-notated key names. Attached is a real-world use-case of where this is failing.3
22USERGRID-161Some required fields should not be required on the users page of the Admin PortalOn the users page of the Admin portal, there are a several fields that are marked as required but should not be.3
23USERGRID-178Lack of Documentation on available features and REST APIsThere is not a proper documentation for available features and REST APIs. It is bit time consuming to do things with Apache Usergrid without documentation. 3
24USERGRID-179Java documentations are misleading the user(developer).Java classes in Usergrid SDKs still referring apigee SDKs classes. ex: Through the method explanation of logOutAppUserAsync() method in Client.java of Usergrid android SDK, referring to a class called "DataClient". There is not a class called "DataClient" in Usergrid android SDK, while apigee SDK has. Therefore it is difficult to figure out by going through the code base.3
25USERGRID-193SaveCollectionMembers in Export does not correctly export all entity dataThe saveConnections() and saveDictionaries() methods are called at the end of SaveCollectionMembers() in file ExportServiceImpl.java but if collection for an entity was null, then it would just return and not export the connections and dictionaries for that particular entity, thus calling these functions in the start of the method ensures, that all connections and dictionaries are always exported.3
26USERGRID-210Fix the "test 10 app limit" test in the two-dot-o branchThis test fails because it fails to account for default apps created by the test framework. Also, do we need to enforce a 10 app limit in Usergrid? ApplicationsIT.test10AppLimit:84 All elements removed expected:<0> but was:<3> The test method above has been marked as Ignore. To fix this bug you should re-enable the test above and then fix the test to properly check the 10 app limit.3
27USERGRID-211Fix "start paging" in the two-dot-o branchNULL3
28USERGRID-212Fix EventsResourceIT.testEventPostandGetThis test fails in code that is still using Hector. EventsResourceIT.testEventPostandGet:111 » UniformInterface GET http://localho... org.apache.usergrid.persistence.exceptions.QueueException: Neccessary queue bounds not found org.apache.usergrid.mq.cassandra.io.AbstractSearch.getQueueRange(AbstractSearch.java:160) org.apache.usergrid.mq.cassandra.io.NoTransactionSearch.getIds(NoTransactionSearch.java:102) org.apache.usergrid.mq.cassandra.io.NoTransactionSearch.getResults(NoTransactionSearch.java:72) org.apache.usergrid.mq.cassandra.QueueManagerImpl.getFromQueue(QueueManagerImpl.java:404) org.apache.usergrid.rest.applications.events.EventsResource.executeGet(EventsResource.java:67) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) The test method above has been marked as Ignored. To fix this bug, re-enable the test, figure out why it fails and fix it.3
29USERGRID-215Possible bug in usergrid-lib.min.js TypeError: c in nullUsing UG (master) when I turn on firebug in firefox and log in to the UG portal, I see the following console error usergrid-lib.min.js (line 27, col 92) ...==this&&c.$$nextSibling))for(;c!==this&&!(d=c.$$nextSibling);)c=c.$parent}while(... First things first, can anyone else reproduce this? I have no idea about why this is as I have not looked into the JavaScript yet. 3
30USERGRID-216GET an non-exists entity by ID or Name throws 401 if no authorisation is requiredfor example get /sandbox/users/non-exist-username throws 401 error code this is because below code {code} @Provider public class ServiceResourceNotFoundExceptionMapper extends AbstractExceptionMapper<ServiceResourceNotFoundException> { @Override public Response toResponse( ServiceResourceNotFoundException e ) { if ( SubjectUtils.getSubjectUserId() == null ) { return toResponse( UNAUTHORIZED, e ); } else { return toResponse( NOT_FOUND, e ); } } } {code}2
31USERGRID-230unable to run usergrid while off line from internetWhen trying to run the launcher and your not connected to the internet there is a spring bean error its unable to load the spring xsd e.g.: http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd that is on line 22 in usergrid-standalone-context.xml in the launcher module. Ive tried to run from source builds or from a-127 command and Its the same issue.3
32USERGRID-231Default collections are not created by database setupSteps to reproduce: 1) setup a new Usergrid system 2) HTTP get the URL /system/database/setup 3) HTTP get the URL /system/superuser/setup 4) Login to the Portal 5) See that there is only a "roles" collection But there should be these collections: /activities /assets /devices /folders /groups /roles /tokens /users 3
33USERGRID-235Inconsistent missing resource status codeAssuming valid credentials, I get a different status code depending on the authorization method. Bearer token: curl -X GET -i -H "Accept: application/json" -H "Authorization: Bearer XXX" 'http://localhost:8080/test-organization/test-app/dogs/fido' 404 error: Service resource not found Client id & Secret: curl -X GET -i -H "Accept: application/json" 'http://localhost:8080/test-organization/test-app/dogs/fido?client_id=XXX&client_secret=XXX' HTTP/1.1 401 Unauthorized Note: In both cases, the client receives the same body content: {"error":"service_resource_not_found","timestamp":1412358355742,"duration":0,"exception":"org.apache.usergrid.services.exceptions.ServiceResourceNotFoundException","error_description":"Service resource not found"} 3
34USERGRID-236Portal looks for helpJson.json in the wrong placeIf you deploy the portal to a path other than "/" then you will get a 404 or possible a 500 error when the portal loads pages because it will not be able to load helpJson.json.3
35USERGRID-237two-dot-o: Error in Portal after entity createAfter an entity is created, the Portal attempts to load the entity but gets a 500 error from the server because it forms a bad URL with two slashes after the application is specified.3
36USERGRID-238Remove ARS Maquette font from Portalportal/css/arsmarquette/* dist/usergrid-portal/css/arsmarquette/* dist-cov/usergrid-portal/css/arsmarquette/* These fonts are Copyright (c) ARS Type®-Angus R. Shamal, 1999-2010. All rights reserved. Do we have a license to redistribute them?2
37USERGRID-241Index Queue should not depend solely on AWS SQS In the two-dot-o branch the faulty queue implementation has been replaced by a set of Queue interfaces and an AWS SQS implementation of those interfaces. Usergrid should not depend on a commercial service and the Usergrid Launcher should be able to run standalone and without external services. So, what we need is a Queue implementation that we can embed in Usergrid for running JUnit tests that depend on the queue, for running the Usergrid Launcher and (ideally) one that can also be run remotely. Apache Qpid seems to be a good candidate for our default queue implementation.3
38USERGRID-249Can't post on connectionAfter setting up my roles as prescribed, I receive this error in the terminal when trying to create a connection: WARNING: Exception occurred during body skip java.lang.IllegalStateException: Can not skip more bytes than available org.glassfish.grizzly.http.server.io.InputBuffer.skip(InputBuffer.java:600) … I tried issuing this command from the shell in the portal: post /users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes { "data": "Learn Usergrid�? } and post /users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes [{ "data": "Learn Usergrid"}] Either command returns: /users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes { "action": "post", "application": "f2b952fa-22ee-11e4-9b4b-e9ea3d610fab", "params": { "access_token": [ "YWMtmZWU0iQUEeScLt1PgUhfegAAAUf7J0uW32RkTiYpwSNVOHBVAtmkMnjFT3s" ] }, "path": "/users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes", "uri": "http://localhost:8080/test.2/note-pad/users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes", "entities": [], "timestamp": 1408063432982, "duration": 6, "organization": "test.2", "applicationName": “note-pad" } However, no entities are being created. I’ve also tried using curl with the same results: curl -H "Authorization: Bearer YWMt2j3JaCQLEeSr1fvx65wFRAAAAUf67ffU8cqvWOyiAVXXIOea177UF05Noa8" -X POST -d '[ {"data":"Lear Usergrid"}]' http://localhost:8080/test.2/note-pad/users/me/mynotes I did execute a post /users/2760f03a-22ef-11e4-8ffd-f3f0018b3726/mynotes by itself as well, which made no difference. -charles 3
39USERGRID-262ES cursor errors should be translated for our usersWhen advancing beyond a cursor's end, or passing an invalid cursor, the following error is returned to the user. {code} body= {"error":"elasticsearch_illegal_argument","timestamp":1417654353286,"duration":0,"error_description":"Failed to decode scrollId","exception":"org.elasticsearch.ElasticsearchIllegalArgumentException"} {code} We should catch this error, and return a friendlier error message to our users.3
40USERGRID-263Limit not honored on subsequent requests with cursorsThe following scenario worked in 1.0. We've broken this in 2.0. Steps to reproduce. 1) Perform a query that returns a cursor, a limit may or may not be supplied. curl -X GET "http://localhost:8080/usergrid/test1/users?limit=20" 2) Perform the next page query and change the limit curl -X GET "http://localhost:8080/usergrid/test1/users?limit=40&cursor=cXVlcnlBbmRGZXRjaDsxOzk4NzA6UDFXa08zSVRTdGFXNncwRmZ2VkZudzswOw==" *What should happen* A response with 40 entities, assuming there are 40 to return. *What actually happens* A response with 20 entities, which is from the original query, but not what was requested.3
41USERGRID-2735 Cleanup test harness and create a more declarative test frameworkCurrently our test harnesses are a mess of spaghetti. We need to decouple this mess, and allow us to cleanly define the rules and requirements of our system. I think we need to break it down into the following behaviors. h1. Test Execution Environments h2. Parallel Execution This should be the default behavior. All tests are assumed to be parallel h2. Serial Execution This should be the exception, and only used when parallel execution cannot be accomplished. Tests in this classification should run in a method at a time, with no concurrency h1. External resources h2. Cassandra # Configure the runtime # Pre invocation hook ## Create default implementation of truncating column families. This should be manually added, and not execute by default. # Post invocation hook h2. ElasticSearch # Configure the runtime # Pre invocation hook ## Create default implementation of truncating column families. This should be manually added, and not execute by default. # Post invocation hook h1. Lifecycle A test itself can have numerous lifecycle operations. These should be set up as rules, both class and instance, and should NOT be done via inheritance of abstract tests. This is the old Junit3 way of operating, and needs removed. 8
42USERGRID-280Implement Admin User methods in Management Class for REST testsImplement Admin User methods in Management Class for REST tests. Ensure that AdminUsersIT test works.3
43USERGRID-281Implement Superuser methods in Management Class for REST testsImplement Superuser methods in Management Class for REST tests Implement superuser token methods also3
44USERGRID-286[SPIKE] Research asynch distributed workflowWe need to develop an asynch distributed workflow for things like data cleanup after app/collection deletion.3
45USERGRID-294move cleanup to before tests run in stackmove cleanup to before tests run in stack 5
46USERGRID-316Usergrid Launcher throws exception if "initialize database" uncheckedIf you run the Launcher and you do not check the "Initialize Database On Start" button, then the first time you start the server the Launcher will exit with an error message like this: 2015-01-06 16:03:51,476 ERROR (pool-2-thread-1) [org.apache.cassandra.config.DatabaseDescriptor] - Fatal configuration error org.apache.cassandra.exceptions.ConfigurationException: Cannot locate file:tmp/cassandra.yaml org.apache.cassandra.config.DatabaseDescriptor.getStorageConfigURL(DatabaseDescriptor.java:123) org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:140) org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:132) org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:216) org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447) org.apache.usergrid.launcher.EmbeddedServerHelper$CassandraRunner.run(EmbeddedServerHelper.java:190) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745)3
47USERGRID-324[SPIKE] Prototype a few distributed realtime parallel processing systemsWe need a system to allow us to build distributed/parallel data process flows. Ultimately, this system must have the following requirements at a component level. h1. Software Requirements # Work well within Reactive/Streams logic # Allow easy operationalization. No additional external systems should be required # Allow failover during node failure, and automatic recovery. Processing should not stop/restart because of a failure, it should resume. # Not additional state administration should need to be built. Distributed futures should be an inherent part of the framework. Some examples are the following. # Migrations # Import/Export # Distributed Indexing on heavily connected entities # Post processing deletes # Collection deletes # Application deletes I have the following requirements. # You can define a deployment topology and limit the number of sub processes/workers in the workflow # Ability to reject requests when there is no capacity # Preferably, do not introduce another dependency (like Zookeeper) and deploy it in the stack war file # An easy intuitive interface for programming flows which will work in a single node, or clustered environment # Administrative status, giving the ability to poll operational status via REST for automated monitoring h2. Examples h3. Guaranteed post processing # A write occurs. It is written to Cassandra and a post processing message is triggered. ## This message must be persistent, and allow for recovery during complete failure. ## If there is no capacity for post processing in the system, we should reject this request via back pressure ## The post processing should return a future (or similar) that could be blocked on if the caller chooses to. # The HTTP call is returned to the user # Post process indexing occurs asynchronously h3. Reindex all entities in the system. # Launch a root process. This process emits all application ids within the system. # Child processes receive the application id. For each app, create the index in elastic search, then emit all collections. # Child processes receive the collections and app ids. For each collection, emit the entity id # Child process receives the app, collection, and id. For each entity, get it's edges, and re-index the documents within elastic search. h3. Delete a collection Realtime -> Update collection alias to point to a new internal collection name. Fire delete collection task. Job process # Launch root process. Load previous collection name and emit to 2 child tasks. # Child task 1: Remove every entity of the previous type from elastic search using bulk delete until empty. # Child task 2: Iterate every entity and remove it from Cassandra, as well as it's graph edge. 8
48USERGRID-332Fix ApplicationRequestCounterITNULL3
49USERGRID-333Fix ApplicationResourceITNULL3
50USERGRID-334Fix AssetResourceITNULL3
51USERGRID-335Test Not Passing: CollectionsResourceITNULL2
52USERGRID-336Fix ContentTypeResourceITNULL1
53USERGRID-338Fix OwnershipResourceITFailing the contextualConnectionOwnership test everytime. The others pass.1
54USERGRID-340Fix RegistrationITNULL1
55USERGRID-341Fix BrowserCompatibilityTestNULL3
56USERGRID-342Fix AdminEmailEncodingITNULL1
57USERGRID-343Fix ApplicationRequestCounterITNULL3
58USERGRID-345Fix RetrieveUsersTestNULL1
59USERGRID-346Fix MatrixQueryTestsNULL3
60USERGRID-347Fix AccessTokenITNULL3
61USERGRID-348Update ExportResourceIT to new REST test frameworkNULL3
62USERGRID-349Fix PropertiesResourceITNULL3
63USERGRID-350Cannot build Android SDKI cannot build the Android SDK in Usergrid 1.0.1. Printouts with build_sdk_zip.sh modified with "-U" and "-e" options added ("mvn clean install -U -e"): ubuntu@ubuntu-VirtualBox:~/Downloads/incubator-usergrid-master/sdks/android$ ./build_release_zip.sh 0.0.8 [INFO] Error stacktraces are turned on. [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/usergrid/usergrid/1.0.0/usergrid-1.0.0.pom [ERROR] The build could not read 1 project -> [Help 1] org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM: Could not find artifact org.apache.usergrid:usergrid:pom:1.0.0 in central (http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 27, column 10 org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:364) org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:672) org.apache.maven.DefaultMaven.getProjectsForMavenReactor(DefaultMaven.java:663) org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:250) org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) org.apache.maven.cli.MavenCli.main(MavenCli.java:157) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) [ERROR] [ERROR] The project org.apache.usergrid:usergrid-android:0.0.8 (/home/ubuntu/Downloads/incubator-usergrid-master/sdks/android/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Could not find artifact org.apache.usergrid:usergrid:pom:1.0.0 in central (http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 27, column 10 -> [Help 2] org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact org.apache.usergrid:usergrid:pom:1.0.0 in central (http://repo.maven.apache.org/maven2) org.apache.maven.project.ProjectModelResolver.resolveModel(ProjectModelResolver.java:159) org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:817) org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:669) org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:307) org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:411) org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:380) org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:344) org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:672) org.apache.maven.DefaultMaven.getProjectsForMavenReactor(DefaultMaven.java:663) org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:250) org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) org.apache.maven.cli.MavenCli.main(MavenCli.java:157) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.apache.usergrid:usergrid:pom:1.0.0 in central (http://repo.maven.apache.org/maven2) org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:459) org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:239) org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:295) org.apache.maven.project.ProjectModelResolver.resolveModel(ProjectModelResolver.java:155) ... 21 more Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.apache.usergrid:usergrid:pom:1.0.0 in central (http://repo.maven.apache.org/maven2) org.eclipse.aether.connector.wagon.WagonRepositoryConnector$6.wrap(WagonRepositoryConnector.java:1012) org.eclipse.aether.connector.wagon.WagonRepositoryConnector$6.wrap(WagonRepositoryConnector.java:1004) org.eclipse.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:725) org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException ubuntu@ubuntu-VirtualBox:~/Downloads/incubator-usergrid-master/sdks/android$ 3
64USERGRID-360Fix ImportIT REST testsNULL2
65USERGRID-361Create REST test for app deleteNULL2
66USERGRID-371Ensure that all services that use amazon keys are using the same identifier for those amazon keys.Currently the way notifications looks for access and secret keys is different from the way that export and import services look for keys. This needs to be changed so that we all look for the same name of the keys.3
67USERGRID-372Usergrid ignores the IF HTTP headersI want to make sure that I don't override changes by accident, so I want to use the [{{If-Unmodified-Since}} HTTP header|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.28]. I expect that the server will give me a 412 (Precondition Failed) error when the data where changed in the meantime. However nothings happens and the data will been always overridden. There is no way to detect changes, even a check before writing could cause a data loss. That should been fixed IMHO. Also the [{{If-Modified-Since}} HTTP header|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.25] would be great to have, I know that I can do that with the CQL too, but a 304 response is faster to detect than a empty result body.3
68USERGRID-375Implement test for resume importNULL3
69USERGRID-379SQS QueueManager implementationThe Usergrid QueueManager is pluggable. We would like to add an SQS implementation.1
70USERGRID-380Cannot delete data with a non admin roleI have a collection which contains data which cannot been deleted from a regular user. I setup this set of permissions: get,put,post,delete:/users/${user}/* delete:/users/${user}/entered/day/** Insert: POST /org/app/users/foobar/entered/day {"value":42} Delete: DELETE /org/app/users/foobar/entered/day/<<uuid>> I can only post/put/get data which belong to the current user, but I cannot delete the data at all, with the admin role this works fine. I think that is a bug. Please check that. 3
71USERGRID-3813 Change serialization format that is used to store entities in CassandraNULL3
72USERGRID-382Clean up migration code (move from Core to CP)Refactor the data migrations to move from Core to Core Persistence for Entity Storage 3
73USERGRID-383Create Service integration tests for ImportCreate Service integration tests - error checking - send bad data - check http status codes - check exception codes 5
74USERGRID-384Get import to work on a distributed system1. write code and tests 2. test on a distributed system3
75USERGRID-385Change import/export format to support multiple collectionsNULL5
76USERGRID-386Change the path for getting status for Import jobNULL2
77USERGRID-387Migrate Import service tests to rest tests to better emulate end to end testingMigrate relevant ImportCollectionIT and ImportServiceIT tests to the rest tier such that we have better end to end coverage of the system. Having all of our integration tests in the service tier means that there is still room for error when we go back up into the REST tier.3
78USERGRID-388Fix Import Resource endpoint to use services to lookup import informationIt's not possible to perform standard collection/connection walking from org/app/import endpoints. Fix this using overridden services. 3
79USERGRID-390Issues with paging in the query system need to be addressed by meaningful changesA recently discovered bug in searching connections has uncovered some incomplete code in the reducer. We're currently mixing 2.0 impl with 1.0. I will be removing old unused 1.0 code, as well as refactoring some of our 2.0 code to be easier to maintain. 8
80USERGRID-393Create Batch Spike for Elastic Search (Todd)NULL3
81USERGRID-395Reset Superuser password -> Create flag to set superuser access to usergrid instancesWe have a superuser endpoint that only works with a system access token. The problem is it can only be accessed with a system access token! Thus when we start up a tomcat we want a flag that can set the superuser access. Also fix the tools so that we can use the superuser admin tool from the command line.5
82USERGRID-396Create UniqueIndexCleanup tool that works with 2.0Current instance of tools don't work correctly with 2.0. This should have a RESTful endpoint that is trigged with a PUT. We should have something like this endpoint. Check all unique values across the entire system => PUT /system/database/unqiuecheck Check all unique values across a single applications => PUT /system/database/uniquecheck/[appid] Check all unique values in a single collection => PUT /system/database/uniquecheck/[appid]/[collectionname] Check a specific unique value for a collection. PUT=> system/database/uniquecheck/[appid]/[collectionname]/[unique value] The unique check will need to audit the unique index, and ensure that the unique value exists for the entity specified. If the entity does not exist, they should be removed. I think we should implement specific property cleanup first, then implement the other operations as an input of Observables 3
83USERGRID-398Create interface layout of new graph traversal read framework and write initial testsNULL3
84USERGRID-399Create interfaces for results processing and write intial testsNULL3
85USERGRID-400Implement read interfacesNULL5
86USERGRID-401Implement write interfacesNULL5
87USERGRID-402[SPIKE] Remove old dictionary calls with Hector and migrate to the map moduleNULL1
88USERGRID-403[SPIKE] Migrate counters from Hector to AstyanaxWe need to migrate our counters from Hector to a more reliable counter framework. This may initially be moving just the hector client to Astyanax, then migrating to something else in a later ticket. 3
89USERGRID-405Fix hotspot from continuous writes of entities (Shawn)Currently, performing a continuous PUT in 2.0 under heavy load causes a hotspot in our cassandra data. Cause: # Under load, entities can be PUT continuously # Asynchronous cleanups run and delete previous versions # These versions are retained in cassandra for long periods of time. This causes severe row bloating before compaction occurs. Solution: For entity data, we only care about the current max version. We should change this column family to store only the maximum data format. We will need to keep the log of previous versions, so that we can bring ES into a consistent state 1
90USERGRID-406HTTP POST of Entity fails under heavy loadWhen running UG2 under heavy we sometimes see the exception below when a new Entity is created by an HTTP POST. The problem seems to be that we write the Entity. Then when we add-to-collection and validate the Entity, we read the Entity but it is not ready for us to read yet -- so the read fails. Instead of re-reading the Entity, we should pass the Entity that we created down into the add-to-collection and validate methods. We should not need to read the Entity in the processing of the request that created it. java.lang.NullPointerException org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:856) org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:842) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsMap(CpEntityManager.java:1094) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsSet(CpEntityManager.java:1017) org.apache.usergrid.services.AbstractService.getConnectingTypesSet(AbstractService.java:505) org.apache.usergrid.services.AbstractService.importEntity(AbstractService.java:364) org.apache.usergrid.services.AbstractService.importEntity(AbstractService.java:333) org.apache.usergrid.services.AbstractCollectionService.postCollection(AbstractCollectionService.java:394) org.apache.usergrid.services.AbstractService.invokeCollection(AbstractService.java:720) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:634) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executePost(ServiceResource.java:401) sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)3
91USERGRID-409Enable system properties for customizable keyspacesKeyspaces should be customizable for the usergrid system. We need to implement the ability to set the keyspace based on system properties, as well as the current defaults. We should implement this in the following way. # Add these properties to the Cassandra Fig (or create a new fig configuration) # Add the fig to the injector # Reference the injector and the fig in the current EntityManager and CassandraService setup classes3
92USERGRID-412Random Tasks, add tasks to comments with timeNULL5
93USERGRID-413Improve Exception when Query String cannot be parsedWhen the Query String has an issue the exception response is confusing. If possible, we should return a better response to the caller. SEVERE: The following errors and warnings have been detected with resource and/or provider classes: SEVERE: Conflicting URI templates. The URI template /{applicationId: [A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}} for sub-resource locator public org.apache.usergrid.rest.management.organizations.applications.ApplicationResource org.apache.usergrid.rest.management.organizations.applications.ApplicationsResource.restoreApplicationFromOrganizationByApplicationId(javax.ws.rs.core.UriInfo,java.lang.String) throws java.lang.Exception and the URI template /{applicationId: [A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}} transform to the same regular expression /([A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12})(/.*)? 2015-02-24 20:17:24,185 [http-bio-8080-exec-27] ERROR org.apache.usergrid.rest.exceptions.ThrowableMapper- An uncaught exception occurred during HTTP invocation com.sun.jersey.spi.inject.Errors$ErrorMessagesException com.sun.jersey.spi.inject.Errors.processErrorMessages(Errors.java:170) com.sun.jersey.spi.inject.Errors.postProcess(Errors.java:136) com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:199) com.sun.jersey.server.impl.application.WebApplicationImpl.getUriRules(WebApplicationImpl.java:564) com.sun.jersey.server.impl.application.WebApplicationContext.getRules(WebApplicationContext.java:244) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722) 2015-02-24 20:17:24,186 [http-bio-8080-exec-27] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- com.sun.jersey.spi.inject.Errors.ErrorMessagesException Server Error (500) com.sun.jersey.spi.inject.Errors$ErrorMessagesException com.sun.jersey.spi.inject.Errors.processErrorMessages(Errors.java:170) com.sun.jersey.spi.inject.Errors.postProcess(Errors.java:136) com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:199) com.sun.jersey.server.impl.application.WebApplicationImpl.getUriRules(WebApplicationImpl.java:564) com.sun.jersey.server.impl.application.WebApplicationContext.getRules(WebApplicationContext.java:244) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722)3
94USERGRID-414Set Build Version to Java 8We're significantly behind our in our RX java implementation. Before work on the EM/RM refactor work starts, we should upgrade to Java 8 to clean up our functional syntax, as well as upgrade to the latest RxJava for performance improvements. We should evaluate missing functions we use in RX as Java 8 streams. If we can re-create our operations as Java 8 streams, we should evaluate the performance of RX Java vs Java 8 streams, and pick the most performant solution.3
95USERGRID-415Consider Not Logging this ExceptionThis exception probably shouldn't be logged at ERROR. Please consider logging it at DEBUG or not at all. 2015-02-25 02:35:55,795 [http-bio-8080-exec-2] ERROR org.apache.usergrid.services.AbstractCollectionService- Entity 7 unable to be created in collection saparticles org.apache.usergrid.persistence.exceptions.DuplicateUniquePropertyExistsException: Entity saparticle requires that property named name be unique, value of 000000000000226126-2538 exists org.apache.usergrid.corepersistence.CpEntityManager.handleWriteUniqueVerifyException(CpEntityManager.java:2556) org.apache.usergrid.corepersistence.CpEntityManager.batchCreate(CpEntityManager.java:2483) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:334) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:280) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:266) org.apache.usergrid.corepersistence.CpRelationManager.createItemInCollection(CpRelationManager.java:746) org.apache.usergrid.corepersistence.CpEntityManager.createItemInCollection(CpEntityManager.java:1325) org.apache.usergrid.services.AbstractCollectionService.postCollection(AbstractCollectionService.java:369) org.apache.usergrid.services.AbstractService.invokeCollection(AbstractService.java:720) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:634) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executePost(ServiceResource.java:401) sun.reflect.GeneratedMethodAccessor135.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722) 2015-02-25 02:35:55,799 [http-bio-8080-exec-2] ERROR org.apache.usergrid.services.AbstractCollectionService- Entity 8 unable to be created in collection saparticles org.apache.usergrid.persistence.exceptions.DuplicateUniquePropertyExistsException: Entity saparticle requires that property named name be unique, value of 000000000000230926-3219 exists org.apache.usergrid.corepersistence.CpEntityManager.handleWriteUniqueVerifyException(CpEntityManager.java:2556) org.apache.usergrid.corepersistence.CpEntityManager.batchCreate(CpEntityManager.java:2483) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:334) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:280) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:266) org.apache.usergrid.corepersistence.CpRelationManager.createItemInCollection(CpRelationManager.java:746) org.apache.usergrid.corepersistence.CpEntityManager.createItemInCollection(CpEntityManager.java:1325) org.apache.usergrid.services.AbstractCollectionService.postCollection(AbstractCollectionService.java:369) org.apache.usergrid.services.AbstractService.invokeCollection(AbstractService.java:720) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:634) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executePost(ServiceResource.java:401) sun.reflect.GeneratedMethodAccessor135.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722)1
96USERGRID-416SPIKE: Profile Usergrid SearchNULL3
97USERGRID-418Fix system/database/setup to swallow "test-app" already exists error.Fix the endpoint to return ok even if the test-app already exists. This isn't really an error and should be swallowed so we don't confuse users. 5
98USERGRID-419Cannot create collections due to new App Delete featureerror_description: "Cannot restore. Deleted Application not found: 7a7624db-bc61-11e4-a2af-12b4f5e28868"
exception: "org.apache.usergrid.persistence.exceptions.EntityNotFoundException" seems like the app restore is intercepting the path that is used to create collections from the UI Request URL:https://example.com/appservices/apigee-jwest/sandbox/?access_token=YWMtefpnELx7EeSlLXug... or the UI isn't right3
99USERGRID-422Create test that proves the issue before making any rest tier changesMake a rest test that shows when calling system/database/setup twice returns an exception.1
100USERGRID-423Added rest tests for system/* endpointsCurrently there are no rest tests that check that the system/database/setup completed correctly. We use test utils that automatically call the service endpoints. We should implement a way to disable/move it so that we can check that the system is being properly initialized using rest tier calls.5
101USERGRID-425[SPIKE] Verify that during an ES re-index, existing documents are force updatedWhen running a reindex last night, documents that appeared in ES were not returned via search in the CandidateResultSet. Deleting the document from ES and then re-indexing resolved the issue. We need to ensure we're updating existing documents that exist in ES during a re-index to force an index update for existing entities. 3
102USERGRID-430Image assets not displaying correctly in SafariOn 1.0 and possibly 2.0. Images display in Chrome, not in Safari Sample of asset data: { "action" : "get", "application" : "50e760c0-99c7-11e4-ac8f-1fb9e891d8f1", "params" : { "access_token" : [ "YWMt62s39LynEeSmFpu1j4AJLwAAAUvjEnPfNEqWy3D0BylnuXAgSHYDP9kvjok\"/" ] }, "path" : "/menuitems", "uri" : "https://api.usergrid.com/fdsafdsa/fdsafdsa/menuitems", "entities" : [ { "uuid" : "03b2686a-a83e-11e4-99f7-b3672594829b", "type" : "menuitem", "created" : 1422594503142, "modified" : 1425255878426, "briefDescription" : "Lorem ipsum dolor sit er elit lamet, consectetaur cillium adipisicing pecu", "detailedDescription" : "This is where the details about this menu item are shown.", "endDate" : 4102473540, "file-metadata" : { "content-type" : "image/png", "etag" : "\"3433eecb1b4fa296c21c9ad8048b4194\"", "last-modified" : 1422594503774, "content-length" : 20542, "checksum" : "3433eecb1b4fa296c21c9ad8048b4194" }, "menuName" : "Pancakes with Raspberries", "metadata" : { "connecting" : { "following" : "/menuitems/03b2686a-a83e-11e4-99f7-b3672594829b/connecting/following" }, "path" : "/menuitems/03b2686a-a83e-11e4-99f7-b3672594829b", "connections" : { "has" : "/menuitems/03b2686a-a83e-11e4-99f7-b3672594829b/has" } }, "prices" : { "priceDescription" : "reg", "price" : 5.5 }, "startDate" : 1388592000 } ], "timestamp" : 1425418403222, "duration" : 10, "organization" : "fdsafdsa", "applicationName" : "fdsafdsa" }5
103USERGRID-432Remove redundant createIndex callsNULL3
104USERGRID-433[SPIKE] SMTP server that starts with the JVM for email rest verificationCurrent email verification is hard to do and requires some messy code. See if there is a better way to integrate that into the rest layer tests.3
105USERGRID-434Rest Test Framework isn't correctly setting up the test accountIn AdminUsersIT we have a test that fails due to the test account not being created when the system spins up. Ideally the test account should be created3
106USERGRID-435Admin Portal - connections listing on data page throws errorSteps to reproduce: 1. Create entities in two different collections 2. Connect them 3. Read the connection in the data explorer POST /reviews {"name":"myreview"} POST /users {"username":"fred"} POST /users/fred/wrote/reviews/myreview GET /users/fred/wrote/reviews/ <== this call will give the error This happens because the data explorer tries to make a /indexes call: GET /users/fred/wrote/reviews/indexes Which returns a 500 error3
107USERGRID-436Call to /indexes produces a 500 error if used with connectionsSteps to reproduce: 1. Create entities in two different collections 2. Connect them 3. Do a GET on the connection with indexes POST /reviews {"name":"myreview"} POST /users {"username":"fred"} POST /users/fred/wrote/reviews/myreview GET /users/fred/wrote/reviews/indexes <== this call will give the error {"error":"null_pointer","timestamp":1425074980254,"duration":0,"exception":"java.lang.NullPointerException"} 3
108USERGRID-437Fix the GET /management/users/<userid> endpoint permissionsgo to the management resource that handles getting the management user and allow them to be retrieved if you have more open permissions. Such that it mirrors the PUT entity endpoint. 3
109USERGRID-438SPIKE: UnexpectedEOF ErrorThis happens occasionally under very light load: {u'fault': {u'faultstring': u'Unexpected EOF', u'detail': {u'errorcode': u'messaging.adaptors.http.UnexpectedEOF'}}} This is not heavy load at all. Will look for the server-side stack trace.3
110USERGRID-439Scheduler run failed when system idlingThis error happens consistently in some clusters when the system is largely idle: 2015-02-27 23:19:40,060 [JobSchedulerService] ERROR org.apache.usergrid.batch.service.JobSchedulerService- Scheduler run failed, error is org.apache.usergrid.persistence.exceptions.QueueException: Unable to obtain a lock on queue '/jobs' after '5'seconds org.apache.usergrid.mq.cassandra.io.ConsumerTransaction.getResults(ConsumerTransaction.java:206) org.apache.usergrid.mq.cassandra.QueueManagerImpl.getFromQueue(QueueManagerImpl.java:412) org.apache.usergrid.batch.service.SchedulerServiceImpl.getJobs(SchedulerServiceImpl.java:164) org.apache.usergrid.batch.service.JobSchedulerService.runOneIteration(JobSchedulerService.java:111) com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:170) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:722)3
111USERGRID-442Investigate ElasticSearch timeout such that a long-running GC does not pause all ES and TomcatI have seen as many as 16 tomcat nodes pausing at the same time across the cluster. I took a stack trace and have attached it. This one jumped out to me: "http-bio-8080-exec-2" daemon prio=10 tid=0x00007f0cbc003800 nid=0x20ff waiting on condition [0x00007f0d370dd000] java.lang.Thread.State: WAITING (parking) sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007ee1de4d8> (a org.elasticsearch.common.util.concurrent.BaseFuture$Sync) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:274) org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113) org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:45) org.apache.usergrid.persistence.index.impl.EsEntityIndexImpl.search(EsEntityIndexImpl.java:357) org.apache.usergrid.corepersistence.CpRelationManager.searchCollection(CpRelationManager.java:963) org.apache.usergrid.corepersistence.CpEntityManager.searchCollection(CpEntityManager.java:624) org.apache.usergrid.corepersistence.CpEntityManagerFactory.lookupApplication(CpEntityManagerFactory.java:433) org.apache.usergrid.rest.organizations.OrganizationResource.getApplicationByName(OrganizationResource.java:137) sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.dispatch(SubLocatorRule.java:197) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.invokeSubLocator(SubLocatorRule.java:183) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:110) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) - locked <0x00000007eda23278> (a org.apache.tomcat.util.net.SocketWrapper) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722) and Todd called out this one: "collectiontasks-18" daemon prio=10 tid=0x00007f0cfc004000 nid=0x21b7 waiting on condition [0x00007f0cb87c6000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007ed32ccf0> (a org.elasticsearch.common.util.concurrent.BaseFuture$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:274) at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113) at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:45) at org.apache.usergrid.persistence.index.impl.EsEntityIndexImpl.deletePreviousVersions(EsEntityIndexImpl.java:533) at org.apache.usergrid.corepersistence.events.EntityVersionCreatedHandler.versionCreated(EntityVersionCreatedHandler.java:67) at org.apache.usergrid.persistence.collection.impl.EntityVersionCreatedTask.fireEvents(EntityVersionCreatedTask.java:97) at org.apache.usergrid.persistence.collection.impl.EntityVersionCreatedTask.call(EntityVersionCreatedTask.java:83) at org.apache.usergrid.persistence.collection.impl.EntityVersionCreatedTask.call(EntityVersionCreatedTask.java:38) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)3
112USERGRID-443Implement Windows Raw Push Notifications for 1.0Support for Raw Push Notifications for Windows platform needs to be added to the 1.0 code base. A separate task can be created/assigned for 2.0.3
113USERGRID-444Test Remove routing by Entity Id in ESWhen we added routing to ES, we were experiencing ES hotspots. This appears to have been caused by uneven shard allocation. We should remove the routing on new indexes, and test this under load.3
114USERGRID-445Create an index merge operationThere is currently no way to remove older indexes and retain only a single index after allocating a new index. We can build this functionality out of our index rebuild, but by adding an upper bound to stop indexing. This should perform the following high level operations. # Find the lowest entity version in the current write alias # Perform an index rebuild on all collections, stopping at the lowest version from the previous step # After a rebuild has been complete, remove all indexes that are not on the current write alias8
115USERGRID-446Removed unused column familiesThere are a lot of remnant column families from 1.0 that are causing bloat in our system. We need to audit each 1.0 column family. If it is no longer used, it should be removed. If it is more complicated to remove these column families (such as queues) we'll need to open tickets for them and address them later. 5
116USERGRID-447Fix data migration format lifecycle on initial setupDuring our initial setup, we perform the following phases. # Setup keyspaces and column families # Set up the root application Admins then have to run the migration via PUT /system/migrate/run This causes an unnecessary migration from V0 to current. Instead, on our initial setup in CPSetup.java, we should perform the following. # Create keyspaces and column families # Perform the data migration, which will simply set the current max version BEFORE data is written # Write the initial root application state1
117USERGRID-448Remove redundant appinfos collections in ManagementServiceImplThere is a flaw in the two-dot-o CpEntityManagerFactory. The factory stores a collection of "appinfo" type entities, but the ManagementServiceImpl stores a redundant collection of "application_info" entities. The problem becomes evident when you try to delete an application. The application will only be deleted from the "appinfos" collection. When you call the management org/apps end-point you will still see the application because the end-point uses the "application_infos". To fix this: - Ensure that only one collection is stored - Add code to migrate the existing app information collections 3
118USERGRID-449Admin Portal - ${user} permissions not allowed, but they should beIn the admin portal, if you try to enter a permission like this: /users/{$user}/has/stuff/mystuff It gives this error: "Path must begin with a slash, path only allows: /, a-z, 0-9, dot, and dash, paths of the format: /path/ or /path//path are not allowed"3
119USERGRID-450Invalidate token after it has been used to activate a new user accountOnce token has been used to activate an account, it should be invalidated.3
120USERGRID-451Upgrade rx to latest versionNULL3
121USERGRID-454Add metrics to cassandra and ES callsWe want to add a way for UG to get metrics about calls being made to Cassandra and ES so we can measure where we have performance bottlenecks. In order to accomplish this we need to wrap our Cassandra execution calls with graphite timers. We could also set timers to check when Cassandra calls are being started or loaded up. For ES we already have timers around index updates and index deletes. We just want to check the other ES calls ( index creations, delete by queries and the rest )5
122USERGRID-455NPE Under LoadNPE under load. Please investigate. java.lang.NullPointerException org.apache.usergrid.services.ServiceContext.getPath(ServiceContext.java:118) org.apache.usergrid.services.AbstractService.checkPermissionsForEntity(AbstractService.java:1240) org.apache.usergrid.services.AbstractCollectionService.putItemByName(AbstractCollectionService.java:314) org.apache.usergrid.services.AbstractService.invokeItemWithName(AbstractService.java:677) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:628) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceReso****...3
123USERGRID-456Change ElasticSearch mapping behavior to be per indexCurrently it is not possible to share multiple usergrid instances on the same cluster. This is due to our default mapping usage in elasticsearch. We need to change the following. # Stop putting default wildcard mappings based on the index prefix, while this should work, it appears it is not # Change our settings so that when an index is created, the default mapping is applied per-index Also during an index rebuild, if the write_alias exists, we will also want to explicitly update the mapping during this process. This will allow us to do an online rebuild, without having to allocate a new write index and then merge. 3
124USERGRID-457Content length headers are not being sent on POSTS for ping identityNULL3
125USERGRID-460Geospatial/Geolocation query results are not sorted by distanceGeospatial ('within') query results are not sorted by distance. We should sort by distance ascending by default. This is now MongoDB handles the $near queries. Please add a test to confirm sorting is done correctly.3
126USERGRID-461Cursor grows as our number of indexes growIt appears our cursor grows as our index grows. This is causing issues with the request growing too large. To alleviate this issue, I think we should perform the following. ON READ # Receive the cursor from ES # Create a new timeUUID # Put the cursor into a map with an expiration equal to the ES expiration time, using the new timeUUID string as a key # Return the time UUID as a string to the user ON next page # Lookup the the ES cursor from Cassandra # Get the next page of results5
127USERGRID-462[SPIKE] Investigate geo location re-index failureWhen re-indexing entities that have a geo_location on it, the geo_field is missing. We need to investigate why this is happening.3
128USERGRID-465Fix remaining graphite timers in CpEntityManagerRefactor of UG-454 where we move the timers into the constructor instead of being initialized in methods. 1
129USERGRID-466Rejected ES Index jobs are dropped. When the index queue is full and ES rejects index requests Usergrid drops them. The result could be that entities cannot be returned. We should handle this soon. At a minimum, we should put them in an SQS queue or something similar for future handling.5
130USERGRID-471Long Pauses and Errors from ES on deleteByQueryWhen a single thread is doing PUTs against Usergrid, one after the other, ES can time out on deleteByQuery and block the caller. The response time from Usergrid goes from 200ms to ~60s during this time. Note that the error in ES logs indicates a 1 minute timeout. [2015-03-10 17:37:42,947][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,947][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,948][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,948][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,949][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:42,949][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,007][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,008][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,012][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,013][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,072][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery ] [res003sy] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-03-10 17:37:43,073][DEBUG][action.deletebyquery3
131USERGRID-473Add Dead Letter QueueNULL3
132USERGRID-474Update cloudformation to use dedicated elasticsearch master nodesWe should update our cloudformation scripts to use a dedicated master node for ES to more closely mirror suggestion production usage during stress testing.1
133USERGRID-478Put entity cleanup and entity delete tasks back to delete by batch.NULL3
134USERGRID-479CLONE - Could not retrieve unique value for field name, unable to verify2015-02-16 05:33:23,590 [http-bio-8080-exec-313] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- java.lang.RuntimeException Server Error (500) java.lang.RuntimeException: Could not retrieve unique value for field name, unable to verify at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:167) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:58) at rx.Observable$12.onNext(Observable.java:4036) at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:61) at rx.Observable$3.call(Observable.java:1551) at rx.Observable$3.call(Observable.java:1546) at rx.Observable.unsafeSubscribe(Observable.java:6839) at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:60) at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: CollectionIoEvent.class at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:98) at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:63) ... 13 more 2015-02-16 05:33:23,590 [http-bio-8080-exec-313] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- Server Error (500): {"error":"runtime","timestamp":1424064803590,"duration":0,"error_description":"Could not retrieve unique value for field name, unable to verify","exception":"java.lang.RuntimeException"} 2015-02-16 05:33:23,600 [http-bio-8080-exec-488] ERROR org.apache.usergrid.rest.exceptions.ThrowableMapper- An uncaught exception occurred during HTTP invocation java.lang.RuntimeException: Could not retrieve unique value for field name, unable to verify at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:167) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:58) at rx.Observable$12.onNext(Observable.java:4036) at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:61) at rx.Observable$3.call(Observable.java:1551) at rx.Observable$3.call(Observable.java:1546) at rx.Observable.unsafeSubscribe(Observable.java:6839) at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:60) at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: CollectionIoEvent.class at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:98)3
135USERGRID-486Create a load testing framework for the query moduleWe need to create a simple stress testing framework for our query module to perform profiling of our indexing scheme. We can do so easily with the following. Re-use our existing gattling cloud formation and usergrid cloud formation Create integration tests that can have an overridden URL to elasticsearch Run these via maven and parallel ssh to get performance results. Use metrics to report throughput, possibly to graphite so we can compare results.5
136USERGRID-488500 Error Message on PUT: version is required to be set for an update operationError is returned with a high load of PUT operations { "duration": 1, "error": "null_pointer", "error_description": "version is required to be set for an update operation", "exception": "java.lang.NullPointerException", "timestamp": 1426565059313 } 5
137USERGRID-490Update index aliases to be created atomicallyCurrently, we move our index aliases in multiple steps. This can lead to inconsistent alias state if fail during one of these operations. Our alias operation should perform a single swap command. We should perform the following during index allocation. # Create the target index successfully # Issue a single http call that will move the write alias, and add the read alias to the new index, per this HTTP command equivalent. http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html#indices-aliases 1
138USERGRID-491Create app has stopped returning new appsWhen creating a new app, no error is returned. However, the app never appears in the portal. Investigate and fix this issue.3
139USERGRID-493Refactor jersey resource calls in rest test framework to use generics.NULL3
140USERGRID-494More than 10 apps will not be returned for organizationsWhen we have more than 10 apps, we cannot return them due to bugs in the PagingResults Iterator. Refactor this to work correctly. In the process, we should also refactor the repair logic to work when reading connected entities as well as collections.1
141USERGRID-497Fix ApplicationResourceITApplicationResourceIT is reporting failures. Investigate root cause as most of them are simply returning back unauthorized.5
142USERGRID-498Broken link to contribution pageOn the usergrid homepage at http://usergrid.incubator.apache.org the "Contribution Guidelines" link points to https://cwiki.apache.org/confluence/display/usergrid/GitHub+Based+Contribution+Workflow but it should should really be https://cwiki.apache.org/confluence/display/usergrid/Usergrid+External+Contributors+Guide3
143USERGRID-501Create migration strategy from multiple index to single indexesNULL8
144USERGRID-502Convert BasicIT over to new rest test frameworkConvert2
145USERGRID-504Querying by application name in rest test framework doesn't work The following call doesn't work management().orgs().organization( clientSetup.getOrganizationName() ) .app().addToPath( clientSetup.getAppName()) but this call does management().orgs().organization( clientSetup.getOrganizationName() ) .app().addToPath( clientSetup.getAppUuid()) The reason being it seems like the application name under the management side also includes the org name as part of its identify but when forming rest calls it doesn't work. Thats why when the name doesn't work the uuid still does . Ideally it should just have the application name as part of its identity. 3
146USERGRID-505Under Load: NullPointerException: version is required to be set for an update operation2015-03-24 17:49:05,432 [http-bio-8080-exec-568] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- java.lang.NullPointerException Server Error (500) java.lang.NullPointerException: version is required to be set for an update operation com.google.common.base.Preconditions.checkNotNull(Preconditions.java:250) org.apache.usergrid.persistence.core.util.ValidationUtils.verifyTimeUuid(ValidationUtils.java:64) org.apache.usergrid.persistence.core.util.ValidationUtils.verifyVersion(ValidationUtils.java:52) org.apache.usergrid.persistence.index.impl.EsEntityIndexBatchImpl.index(EsEntityIndexBatchImpl.java:104) org.apache.usergrid.corepersistence.CpEntityManager.indexEntityIntoCollection(CpEntityManager.java:2990) org.apache.usergrid.corepersistence.CpRelationManager.addToCollection(CpRelationManager.java:708) org.apache.usergrid.corepersistence.CpEntityManager.batchCreate(CpEntityManager.java:2686) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:404) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:348) org.apache.usergrid.corepersistence.CpEntityManager.create(CpEntityManager.java:334) org.apache.usergrid.services.AbstractCollectionService.putItemByName(AbstractCollectionService.java:300) org.apache.usergrid.services.AbstractService.invokeItemWithName(AbstractService.java:677) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:628) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executePutWithMap(ServiceResource.java:371) org.apache.usergrid.rest.applications.ServiceResource.executePut(ServiceResource.java:420) sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722)5
147USERGRID-506Numerous Null Pointer Exceptions on bulk deletesI am doing some bulk deletes and getting many NPE's and errors in the log files. During this time, Tomcat or ELB seems enticed to return many 503's. If this goes on too long the tomcats become unresponsive. Got some stack traces... 2015-03-24 20:01:55,925 [http-bio-8080-exec-5] ERROR org.apache.usergrid.corepersistence.CpEntityManager- Unable to load entity saparticl:03b6b5e4-c906-11e4-821c-651cb62f6d39 java.lang.NullPointerException org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:956) org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:942) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsMap(CpEntityManager.java:1205) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsSet(CpEntityManager.java:1119) org.apache.usergrid.services.AbstractService.getConnectedTypesSet(AbstractService.java:487) org.apache.usergrid.services.AbstractService.importEntity(AbstractService.java:355) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:402) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:414) org.apache.usergrid.services.AbstractCollectionService.deleteItemsByQuery(AbstractCollectionService.java:528) org.apache.usergrid.services.AbstractService.invokeItemsWithQuery(AbstractService.java:703) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:625) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executeDelete(ServiceResource.java:437) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722) 2015-03-24 20:01:56,030 [http-bio-8080-exec-5] ERROR org.apache.usergrid.corepersistence.CpEntityManager- Unable to load entity saparticl:03d8bcca-c906-11e4-bd5d-bdad5573d5fc java.lang.NullPointerException org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:956) org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:942) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsMap(CpEntityManager.java:1205) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsSet(CpEntityManager.java:1119) org.apache.usergrid.services.AbstractService.getConnectedTypesSet(AbstractService.java:487) org.apache.usergrid.services.AbstractService.importEntity(AbstractService.java:355) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:402) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:414) org.apache.usergrid.services.AbstractCollectionService.deleteItemsByQuery(AbstractCollectionService.java:528) org.apache.usergrid.services.AbstractService.invokeItemsWithQuery(AbstractService.java:703) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:625) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executeDelete(ServiceResource.java:437) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:722) 2015-03-24 20:01:56,033 [http-bio-8080-exec-5] ERROR org.apache.usergrid.corepersistence.CpEntityManager- Unable to load entity saparticl:03d8bcca-c906-11e4-bd5d-bdad5573d5fc java.lang.NullPointerException org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:956) org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:942) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsMap(CpEntityManager.java:1205) org.apache.usergrid.corepersistence.CpEntityManager.getDictionaryAsSet(CpEntityManager.java:1119) org.apache.usergrid.services.AbstractService.getConnectingTypesSet(AbstractService.java:505) org.apache.usergrid.services.AbstractService.importEntity(AbstractService.java:364) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:402) org.apache.usergrid.services.AbstractService.importEntities(AbstractService.java:414) org.apache.usergrid.services.AbstractCollectionService.deleteItemsByQuery(AbstractCollectionService.java:528) org.apache.usergrid.services.AbstractService.invokeItemsWithQuery(AbstractService.java:703) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:625) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executeDelete(ServiceResource.java:437) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:601) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve....3
148USERGRID-507Many Histrix timeouts though Cassndra is not under loadCassandra is not under heavy load but I am seeing a high number of Hystrix timeouts. Should we increase connection pool size? Is there an operational setting that can help this? 2015-03-24 18:34:17,420 [graphTaskExecutor-3] ERROR org.apache.usergrid.persistence.core.task.NamedTaskExecutorImpl- Unable to execute task. Exception is com.netflix.hystrix.exception.HystrixRuntimeException: HystrixCassandra$1 timed-out and no fallback available. at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1646) at com.netflix.hystrix.HystrixCommand.access$1900(HystrixCommand.java:103) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$1.run(HystrixCommand.java:1023) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:41) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:37) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable.run(HystrixContextRunnable.java:57) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$2.tick(HystrixCommand.java:1047) at com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:627) at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:522) at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:431) at org.apache.usergrid.persistence.core.hystrix.HystrixCassandra.user(HystrixCassandra.java:69) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.advance(MultiRowColumnIterator.java:187) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.hasNext(MultiRowColumnIterator.java:124) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardsColumnIterator.hasNext(ShardsColumnIterator.java:65) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.NodeShardAllocationImpl.auditShard(NodeShardAllocationImpl.java:208) at sun.reflect.GeneratedMethodAccessor193.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37) at com.sun.proxy.$Proxy92.auditShard(Unknown Source) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:368) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:319) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: java.util.concurrent.TimeoutException ... 25 more 2015-03-24 18:34:17,420 [graphTaskExecutor-3] ERROR org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl- Unable to execute audit for shard of {} com.netflix.hystrix.exception.HystrixRuntimeException: HystrixCassandra$1 timed-out and no fallback available. at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1646) at com.netflix.hystrix.HystrixCommand.access$1900(HystrixCommand.java:103) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$1.run(HystrixCommand.java:1023) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:41) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:37) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable.run(HystrixContextRunnable.java:57) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$2.tick(HystrixCommand.java:1047) at com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:627) at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:522) at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:431) at org.apache.usergrid.persistence.core.hystrix.HystrixCassandra.user(HystrixCassandra.java:69) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.advance(MultiRowColumnIterator.java:187) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.hasNext(MultiRowColumnIterator.java:124) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardsColumnIterator.hasNext(ShardsColumnIterator.java:65) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.NodeShardAllocationImpl.auditShard(NodeShardAllocationImpl.java:208) at sun.reflect.GeneratedMethodAccessor193.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37) at com.sun.proxy.$Proxy92.auditShard(Unknown Source) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:368) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:319) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: java.util.concurrent.TimeoutException ... 25 more 2015-03-24 18:34:17,420 [graphTaskExecutor-3] ERROR org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl- Unable to perform audit. Exception is com.netflix.hystrix.exception.HystrixRuntimeException: HystrixCassandra$1 timed-out and no fallback available. at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1646) at com.netflix.hystrix.HystrixCommand.access$1900(HystrixCommand.java:103) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$1.run(HystrixCommand.java:1023) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:41) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:37) at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable.run(HystrixContextRunnable.java:57) at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$2.tick(HystrixCommand.java:1047) at com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:627) at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:522) at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:431) at org.apache.usergrid.persistence.core.hystrix.HystrixCassandra.user(HystrixCassandra.java:69) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.advance(MultiRowColumnIterator.java:187) at org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.hasNext(MultiRowColumnIterator.java:124) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardsColumnIterator.hasNext(ShardsColumnIterator.java:65) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.NodeShardAllocationImpl.auditShard(NodeShardAllocationImpl.java:208) at sun.reflect.GeneratedMethodAccessor193.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37) at com.sun.proxy.$Proxy92.auditShard(Unknown Source) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:368) at org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupCompactionImpl$ShardAuditTask.call(ShardGroupCompactionImpl.java:319) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: java.util.concurrent.TimeoutException ... 25 more3
149USERGRID-508Delete call with a limit of X returns >X reslutsWhen doing a delete call with ql=select * and lmit=100, >100 results are returned. This may or may not imply that >100 entities were deleted.3
150USERGRID-509Cardinality of the EntityCollectionManager is incorrect and may need refactoredAn entity can only exist in a single scope. However, this makes it impossible to batch fetch entities from multiple scopes in a single network request. This interface needs changed. Our EntityCollectionManagerFactory needs to change to return an EntityCollectionManager by ApplicationScope. Further read/write requests would then need to take an encapsulating CollectionScope. This would allow us to batch write and batch fetch multiple entities across different collections in a single request.5
151USERGRID-510Write/Review Test Framework GuideTask for writing up how the current REST test framework behaves and what expectations are for new tests. We should have 90 minutes web conf where we record how we go about doing the tests, to review a 'good' test, a 'bad' test and a test that is not on the current framework.1
152USERGRID-511Refactor current collection and connection code to quickly fix our current read issuesThe refactor into our new functional framework is taking longer than anticipated. We should quickly adapt our current 2.0 implementation to resolve this issue, then fix it properly in the USERGRID-494 issue.3
153USERGRID-514Limit the number of threads/resources an index rebuild can consumeThe rebuilding of an index seems to run flat-out and consumes lots of resources in the process. It would be nice to have some controls on this.3
154USERGRID-515[SPIKE] Creating collections in the portal throws an errorThe '+' button to create a collection in the UI does not work. In Chrome dev tools it looks like it is doing a PUT to this URL: https://{host}/{org}/{app}/?access_token={token} with this request data: {"metadata":{"collections":{"foo":{}}}} And the response looks like this: duration: 6 error: "entity_not_found" error_description: "Cannot restore. Deleted Application not found: 5ab1ef5c-7f4c-11e4-9bb8-0ad113d917ad" exception: "org.apache.usergrid.persistence.exceptions.EntityNotFoundException" timestamp: 1427291983254 Open tickets to do the fixes as necessary1
155USERGRID-516BaaS Queries returning Entities which do not match the input Queryhttps://apigeesc.atlassian.net/browse/APIBAAS-15143
156USERGRID-518Update copyright in footer of Portal to 2015NULL1
157USERGRID-519Configuring Notifier with APNS does not error when a non-p12 certificate is uploaded or usedI uploaded a x509 certificate instead of a p12 certificate for configuring an APS notifier. I experienced the following: 1) The UI did not error out when I uploaded a non p12 certificate. -- Similar to Parse, the console should error out when a non-p12 certificate is attempted to be used. If we accept an x509 certificate the docs and UI page should be updated. 2) When sending a push notification BaaS reports a success in the message history and notification entity but a push notification was not received -- It seems that this is not possible to have worked since it was not a p12 certificate. It looks as if there could be an exception getting swallowed. 3) I cannot delete the notifiers. -- Dont know if this is related to the certificate or not 3
158USERGRID-521Ability to view/replace an existing APN certificateCustomer requests that we have the ability to view and/or replace an existing APN certificate without having to delete the existing one. This approach would minimize downtime for notification services. Ability to view the certificate, or at more likely only the certificate details) will provide the ability for an operations team to monitor its validity. 1
159USERGRID-527Write external ES test to ensure we get even routing across the ES clusterOne of the problems we have faced is uneven distribution in the ES cluster. We have made changes for document routing that seemed to help, but we are still getting hot nodes in the cluster. We need to have a test that hits ES directly with load as UG would do to test our cluster distribution.3
160USERGRID-528Refactor the Query portions of RelationManager into a command patternUsing Rx and functional programming. Implemented a builder + command pattern to allow us to chain observables together to generate a stream of results.8
161USERGRID-529iOS Push Notification ErrorsSee the following ticket: https://apigeesc.atlassian.net/browse/APIBAAS-1525. 2015-03-31 13:57:37,870 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:37,872 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:37,873 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:37,881 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,080 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,084 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,085 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,093 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,292 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,299 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,300 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,304 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,510 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,511 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,513 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,517 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,723 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,724 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,725 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,729 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,935 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,936 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,938 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:38,941 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:39,146 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection java.nio.channels.ClosedChannelException 2015-03-31 13:57:39,147 ERROR (pool-525-thread-1) [org.apache.usergrid.services.notifications.apns.RejectedAPNsListener] - Failed to register push connection 3
162USERGRID-530Deleting a connection to an Asset should not delete the Asset's dataSteps to reproduce: 1) Create asset entity 2) Create entity 3) Create connection from entity to asset entity 4) Delete connection 5) Asset JSON still present, Asset data is now gone 1
163USERGRID-532Endpoint to get the size of the SQS index queueNULL2
164USERGRID-533REST Test for Create App not returningEnsure we create >11 apps and that it works.2
165USERGRID-534 Load Test with 2.1Test load with the frankenloader in e2e for the upcoming release3
166USERGRID-535[SPIKE] Look into broken tests and open ticketsTake a look at REST and CORE tests to see what needs to be cleaned up to stabilize for upcoming release.3
167USERGRID-536Change our index structure for static mapping and cleanup apiCurrently, our dynamic mapping causes several issues with elastic search. We should change our mapping to use a static structure, and resolve this operational pain. We need to make the following changes. h2. Modify our IndexScope This should more closely resemble the elements of an edge since this represents an edge. It will simplify the use of our query module and make development clearer. This scope should be refactored into the following objects. * IndexEdge - Id, name, timestamp, edgeType (source or target) * SearchEdge - Id, name, edgeType Note: edgeType is the type of the Id within the edge. Does this Id represent a source Id, or does it represent a targetId? The entity to be indexed will implicitly be the opposite of the type specified. I.E if it's a source edge, the document is the target. If it's a target edge, the document is the source. These values should also be stored within our document, so that we can index our documents. Note that we perform bidirectional indexing in some cases, such was users, groups etc. When we do this, we need to ensure that mark the direction of the edge appropriately. h2. Change default sort ordering When sorting is unspecified, we should order by timestamp descending from our index edge. This ensures that we retain the correct edge time semantics, and will properly order collections and connections h2. Remove the legacy query class We don't need the Query class, it has far too many functions to be a well encapsulated object. Instead, we should simply take the string QL, the SearchEdge and the limit to return our candidates. From there, we should parse and visit the query internally to the query logic, NOT externally. h2. Create a static mapping The mapping should contains the following static fields. * entityId - The entity id * entityType - The entity type (from the id) * entityVersion - The entity version * edgeId - The edge Id * edgeName - The edge name * edgeTimestamp - The edge timestamp * edgeType - source | target * edgeSearch - edgeId + edgeName + edgeType It will then contain an array of "fields" Each of these fields will have the following formation. {code} { "name":"[entity field name as a path]", "[field type]":[field value} {code} We will define a field type for each type of field. Note that each field tuple will always contain a single field and a single value. Possible field types are the following. * string - This will be mapped into 2 mapping with multi mappings. It will be a string unanalyzed, and an analyzed string. The 2 fields will then be "string_u" and "string_a". The Query visitor will need to update the field name appropriately * long - An unanalyzed long * double - An unanalyzed double * boolean - An unanalyzed boolean * location - A geolocation field * uuid - A UUID stored as an unanalyzed string The entity path will be a flattened path from the root json element to the max json element. It can be though of as a path through the tree of json elements. We will use a dot '.' to delimit the fields. X.Y.Z for nested objects. Primitive arrays will contain a field object for each element in the array. h2. Indexing When indexing entities, we will no longer modify or prefix field names. They will be inserted into the value exactly as their path appears after lower case. h2. Querying When querying, the "contains" operation for a string will need to use the "string_a" data type. When using =, we will need to use the string_u data type. Each criteria will need to use nested object querying, to ensure the property name and property value are both part of the same field tuple. h3. References Multi Field Mapping: http://www.elastic.co/guide/en/elasticsearch/reference/current/_multi_fields.html Nested Objects: http://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html Nested Object Search: http://www.elastic.co/guide/en/elasticsearch/guide/master/nested-sorting.html 2
168USERGRID-537Write Unique Verify - could not get uniqueValue from the field name2015-04-01 18:57:54,440 ERROR (http-bio-8080-exec-224) AbstractExceptionMapper - Server Error (500): {"error":"hystrix_runtime","timestamp":1427929074440,"duration":0,"error_description":"ConsistentReplayCommand failed and failed retrieving fallback.","exception":"com.netflix.hystrix.exception.HystrixRuntimeException"} 2015-04-01 18:57:54,440 ERROR (hystrix-user-94) WriteUniqueVerify - Could not get uniqueValue from the field name. 2015-04-01 18:57:54,440 ERROR (http-bio-8080-exec-325) ThrowableMapper - An uncaught exception occurred during HTTP invocation com.netflix.hystrix.exception.HystrixRuntimeException: ConsistentReplayCommand failed and failed retrieving fallback. at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1660) at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1341) at com.netflix.hystrix.HystrixCommand.access$2300(HystrixCommand.java:103) at com.netflix.hystrix.HystrixCommand$5.call(HystrixCommand.java:1186) at com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:51) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: Could not retrieve unique value for field name, unable to verify at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.executeStrategy(WriteUniqueVerify.java:207) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.run(WriteUniqueVerify.java:178) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.run(WriteUniqueVerify.java:159) at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1281) ... 7 more Caused by: java.lang.Throwable: Calling Thread included as the last 'caused by' on the chain. at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1033) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:282) at rx.internal.operators.BlockingOperatorToFuture$2.get(BlockingOperatorToFuture.java:111) at com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:623) at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:522) at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:431) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:151) at org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:68) at rx.Observable$12.onNext(Observable.java:4036) at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:61) at rx.Observable$3.call(Observable.java:1551) at rx.Observable$3.call(Observable.java:1546) at rx.Observable.unsafeSubscribe(Observable.java:6839) at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:60) at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) ... 3 more Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: CollectionIoEvent.class at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:98) at rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:63) ... 12 more5
169USERGRID-538Improve uncaught errors' http responseWhen an uncaught exception happens in stack's rest layer, we throw the java error in response to the user. Such responses could just say internal server error. And as [~tnine] suggested, we could add a time uuid to the error response, so we can correlate the error in logs and track the error down for the user. 1
170USERGRID-539Test Not Passing: CPHeadEntityNull in Services testsTests in error: EmailFlowIT.testAppUserActivationResetpwdMail:287 » IllegalArgument cpHeadEnti... EmailFlowIT.testAppUserConfirmationMail:376 » IllegalArgument cpHeadEntity can... OrganizationIT.testCreateOrganization:91 » IllegalArgument cpHeadEntity cannot...3
171USERGRID-541Change the current relation manager impl to and create indexing interfaceadd interface to index appid and entity 1. read entity state out of cass. 2. read all edges 3. if bidirectional -> index from source to target 4. submit batch 5. wait for batch -> return future 5
172USERGRID-542Have Export output password hashes in 1.0Make sure that the export process outputs the password hashes. This will allow people who want to import user accounts able to by using the api. 5
173USERGRID-543Add Queue Length to /status pageAdd queue length (SQS, akka, etc.) to the data returned by /status3
174USERGRID-544[SPIKE] Deadletter - Figure out why some entities are having indexing problemswe have some entities that are having indexing issues, with 6,011 at this time in a dead letter queue. we need to understand why these are having indexing issues.3
175USERGRID-545Endpoint for checking storage consumedAs we move to separate keyspaces for each tenant we need to have an endpoint to monitor storage consumed for the keyspace. This will be followed by the ability to show data consumed in the console and notifications when storage consumed is at X%. 1. add size to entities 2. add size to index. 3. migrate field mapping in index 4. add endpoint for getting storage by app api is management/orgs/orgname/applications/appname/size or management/orgs/orgname/applications/appname/size/collectionName3
176USERGRID-546Ensure SQS Consumers are robust and do not stop consuming messagesIn the past: For some reason the SQS queue consumers stop processing messages. Please investigate, ensure that the code is robust, and add log messages. Current: Test and confirm, review with Jeff and/or Todd that the loop for processing SQS messages is robust.3
177USERGRID-548Allow Export to output admin password hashesNULL5
178USERGRID-549Refactor Org app relationship to remove app info object and use graph to application objectNULL1
179USERGRID-550Create migration for app infos into new formatNULL1
180USERGRID-551[SPIKE] Cross-collection / graph filtering does not workREST calls of this nature do not work: GET /pets;ql=select * where type='cat'/belongsTo/owners;ql=select * where city='Dallas' 5
181USERGRID-552Make number of shards for map configurableCurrently the count of map shards is hard coded. Make this a config value, and change the defaults while retaining the older versions. We need the ability to expand this value, and configure it at runtime.1
182USERGRID-553Modify graph edges to use buckets per shardDuring heavy write load, we are not getting good load distribution across multiple row keys. We need to add row key sharding to our graph edges to avoid causing hotspots in our nodes.5
183USERGRID-555Test Migration of Entity VersionsTest migration from v2 to v3 for the upcoming release3
184USERGRID-556Update Index prefix for management app to use the cassandra keyspace nameWe need to replicate and test the reindex oscillation issue in the e2e environment.1
185USERGRID-559[FIXED?] Add re-index resumeWe need the ability to resume re-indexing. To do this, we should make the AllEntitiesObservable allow for resume by default. This way if the node performing the migration fails, it can be restarted on another node. Note that we also need a way to clear state via REST, so that we can restart our indexing.3
186USERGRID-560Ensure USERGRID-530 is addressed in two-dot-o(-dev)https://issues.apache.org/jira/browse/USERGRID-530. Make sure it gets in 2.03
187USERGRID-561Portal appears to hang when it gets a 401When logging into the portal it appears to hang when a 401 is returned from https://stage-exchange-prod.apigee.net/appservices/management/users/{email}. We should catch this error and handle it better.1
188USERGRID-563[SPIKE] Ability for one Usergrid instance to delegate authentication to another Design forthcoming.3
189USERGRID-566Updating default accesstoken ttl for an app gives 404PUT call to an organization's app to update accesstoken ttl gives 404. The REST call is documented here - http://apigee.com/docs/api-baas/content/changing-token-time-live-ttl This fails in two-dot-o and two-dot-o-dev. The code which handles the api has been updated recently (past month).3
190USERGRID-567Validate External Token for Usergrid Central SSOThe requirement is to have multiple Usergrid systems, each with its own Cassandra cluster, be able to authenticate Admin Users with one central Usergrid system -- giving Admin Users Single-Sign-On (SSO) across all of those systems. We can do this by adding just one new end-point to Usergrid. This Google Doc explains a complete design for "Usergrid Central SSO": https://docs.google.com/document/d/12kXgaYcB6L9JoTyRGn0ZHEMg3vL1LJDqvtnltIBDa1Y/edit?usp=sharing The design is based on earlier work by Ed Anuff and Nate McCall.5
191USERGRID-568Fix push issues close connection issuesNULL3
192USERGRID-570New Apps not in dropdown until you click on Org Administration PageWhen creating apps in the portal sometimes it takes a very long time to reflect them in the portal. Even after /users/me returns the app in the list it is still not reflected in the portal. When you use the '+ Add New App' at the top of the page the newly created app is not reflected in the App dropdown until you click on the Org Administration page. See Jeff to get a URL for a recording of this issue being reproduced. In addition, sometimes the immediate result is 'No Application Access Authorized'.3
193USERGRID-572Add a query parameter to enable the hiding of sensitive information in API resposnesPlease add a setting that can prevent the qparams from being returned in a response, and/or add a filter for accessToken and ClientID/ClientSecret to not be returned in a response.1
194USERGRID-573Fix testCursorFormat in queryIndex ModuleNULL2
195USERGRID-575Disable Admin User signups when external token validation enabledWhen external token validation is enabled (see Usergrid-567) and a Usergrid system is delegating authentication to some other central Usergrid for SSO purposes, then new Admin Users should register ONLY with that central Usergrid. 3
196USERGRID-576Add metrics to external token validation endpointUse the MetricsFactory to add metrics for successful token validations, unsuccessful token validations and local admin user creation. For more about external token validation see Usergrid-5673
197USERGRID-577Make all Cassandra keyspace names configurableThis has already been done in the two-dot-o branch. Fixing it in 1.0 will be a little different. These keyspace names are hard-coded into Java code: - Usergrid - Usergrid_Applications and the Locks keyspace name is hardcoded into a Spring config.3
198USERGRID-578ES Cursors not working in 2.1NULL3
199USERGRID-579Exception when trying to post/retrieve an asset from Amazon S32015-04-19 13:50:15,455 ERROR (http-bio-8080-exec-24) [package org.apache.usergrid.rest.exceptions] - Server Error (500): {"error":"creation","timestamp":1429465815453,"duration":2,"exception":"com.google.inject.CreationException","error_description":"Guice creation errors:\n\n1) No implementation for com.google.common.base.Function<java.lang.String, com.google.common.base.Optional<java.lang.String>> annotated with @org.jclouds.s3.Bucket() was bound.\n while locating com.google.common.base.Function<java.lang.String, com.google.common.base.Optional<java.lang.String>> annotated with @org.jclouds.s3.Bucket()\n for parameter 0 at org.jclouds.s3.blobstore.functions.LocationFromBucketName.<init>(LocationFromBucketName.java:52)\n at org.jclouds.s3.blobstore.config.S3BlobStoreContextModule.configure(S3BlobStoreContextModule.java:60)\n\n2) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound.\n while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>>\n for parameter 7 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43)\n\n3) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound.\n while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>>\n for parameter 7 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore\n for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46)\n\n4) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound.\n while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>>\n for parameter 5 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44)\n\n5) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound.\n while locating org.jclouds.blobstore.domain.Blob$Factory\n for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39)\n while locating org.jclouds.s3.blobstore.functions.ObjectToBlob\n for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43)\n\n6) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound.\n while locating org.jclouds.blobstore.domain.Blob$Factory\n for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39)\n while locating org.jclouds.s3.blobstore.functions.ObjectToBlob\n for parameter 8 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44)\n\n7) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound.\n while locating org.jclouds.blobstore.domain.Blob$Factory\n for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39)\n while locating org.jclouds.s3.blobstore.functions.ObjectToBlob\n for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore\n for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46)\n\n8) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound.\n while locating org.jclouds.s3.domain.S3Object$Factory\n for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38)\n while locating org.jclouds.s3.blobstore.functions.BlobToObject\n for parameter 1 at org.jclouds.aws.s3.blobstore.AWSS3BlobRequestSigner.<init>(AWSS3BlobRequestSigner.java:64)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.bindRequestSigner(AWSS3BlobStoreContextModule.java:51)\n\n9) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound.\n while locating org.jclouds.s3.domain.S3Object$Factory\n for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38)\n while locating org.jclouds.s3.blobstore.functions.BlobToObject\n for parameter 12 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43)\n\n10) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound.\n while locating org.jclouds.s3.domain.S3Object$Factory\n for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38)\n while locating org.jclouds.s3.blobstore.functions.BlobToObject\n for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44)\n\n11) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound.\n while locating org.jclouds.s3.domain.S3Object$Factory\n for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38)\n while locating org.jclouds.s3.blobstore.functions.BlobToObject\n for parameter 1 at org.jclouds.aws.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.<init>(SequentialMultipartUploadStrategy.java:66)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:45)\n\n12) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound.\n while locating org.jclouds.s3.domain.S3Object$Factory\n for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38)\n while locating org.jclouds.s3.blobstore.functions.BlobToObject\n for parameter 12 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85)\n while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore\n for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101)\n at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46)\n\n13) org.jclouds.rest.config.SyncToAsyncHttpApiProvider<org.jclouds.rest.HttpClient, A> cannot be used as a key; It is not fully specified.\n\n14) org.jclouds.rest.config.SyncToAsyncHttpApiProvider<org.jclouds.aws.s3.AWSS3Client, A> cannot be used as a key; It is not fully specified.\n\n15) org.jclouds.rest.RestContext<org.jclouds.aws.s3.AWSS3Client, A> cannot be used as a key; It is not fully specified.\n\n16) No implementation for org.jclouds.rest.HttpClient was bound.\n at org.jclouds.rest.config.BinderUtils.bindHttpApiProvider(BinderUtils.java:107)\n\n17) No implementation for org.jclouds.aws.s3.AWSS3Client was bound.\n at org.jclouds.rest.config.BinderUtils.bindHttpApiProvider(BinderUtils.java:107)\n\n17 errors"} 2015-04-19 13:50:19,598 ERROR (http-bio-8080-exec-25) [package org.apache.usergrid.rest.exceptions] - com.google.inject.CreationException Server Error (500) com.google.inject.CreationException: Guice creation errors: 1) No implementation for com.google.common.base.Function<java.lang.String, com.google.common.base.Optional<java.lang.String>> annotated with @org.jclouds.s3.Bucket() was bound. while locating com.google.common.base.Function<java.lang.String, com.google.common.base.Optional<java.lang.String>> annotated with @org.jclouds.s3.Bucket() for parameter 0 at org.jclouds.s3.blobstore.functions.LocationFromBucketName.<init>(LocationFromBucketName.java:52) at org.jclouds.s3.blobstore.config.S3BlobStoreContextModule.configure(S3BlobStoreContextModule.java:60) 2) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound. while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> for parameter 7 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43) 3) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound. while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> for parameter 7 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46) 4) No implementation for com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> was bound. while locating com.google.common.base.Function<java.util.Set<org.jclouds.s3.domain.BucketMetadata>, org.jclouds.blobstore.domain.PageSet<? extends org.jclouds.blobstore.domain.StorageMetadata>> for parameter 5 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44) 5) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound. while locating org.jclouds.blobstore.domain.Blob$Factory for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39) while locating org.jclouds.s3.blobstore.functions.ObjectToBlob for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43) 6) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound. while locating org.jclouds.blobstore.domain.Blob$Factory for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39) while locating org.jclouds.s3.blobstore.functions.ObjectToBlob for parameter 8 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44) 7) No implementation for org.jclouds.blobstore.domain.Blob$Factory was bound. while locating org.jclouds.blobstore.domain.Blob$Factory for parameter 0 at org.jclouds.s3.blobstore.functions.ObjectToBlob.<init>(ObjectToBlob.java:39) while locating org.jclouds.s3.blobstore.functions.ObjectToBlob for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46) 8) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound. while locating org.jclouds.s3.domain.S3Object$Factory for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38) while locating org.jclouds.s3.blobstore.functions.BlobToObject for parameter 1 at org.jclouds.aws.s3.blobstore.AWSS3BlobRequestSigner.<init>(AWSS3BlobRequestSigner.java:64) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.bindRequestSigner(AWSS3BlobStoreContextModule.java:51) 9) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound. while locating org.jclouds.s3.domain.S3Object$Factory for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38) while locating org.jclouds.s3.blobstore.functions.BlobToObject for parameter 12 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:43) 10) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound. while locating org.jclouds.s3.domain.S3Object$Factory for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38) while locating org.jclouds.s3.blobstore.functions.BlobToObject for parameter 10 at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.<init>(AWSS3BlobStore.java:77) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:44) 11) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound. while locating org.jclouds.s3.domain.S3Object$Factory for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38) while locating org.jclouds.s3.blobstore.functions.BlobToObject for parameter 1 at org.jclouds.aws.s3.blobstore.strategy.internal.SequentialMultipartUploadStrategy.<init>(SequentialMultipartUploadStrategy.java:66) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:45) 12) No implementation for org.jclouds.s3.domain.S3Object$Factory was bound. while locating org.jclouds.s3.domain.S3Object$Factory for parameter 1 at org.jclouds.s3.blobstore.functions.BlobToObject.<init>(BlobToObject.java:38) while locating org.jclouds.s3.blobstore.functions.BlobToObject for parameter 12 at org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore.<init>(AWSS3AsyncBlobStore.java:85) while locating org.jclouds.aws.s3.blobstore.AWSS3AsyncBlobStore for parameter 0 at org.jclouds.aws.s3.blobstore.strategy.internal.ParallelMultipartUploadStrategy.<init>(ParallelMultipartUploadStrategy.java:101) at org.jclouds.aws.s3.blobstore.config.AWSS3BlobStoreContextModule.configure(AWSS3BlobStoreContextModule.java:46) 13) org.jclouds.rest.config.SyncToAsyncHttpApiProvider<org.jclouds.rest.HttpClient, A> cannot be used as a key; It is not fully specified. 14) org.jclouds.rest.config.SyncToAsyncHttpApiProvider<org.jclouds.aws.s3.AWSS3Client, A> cannot be used as a key; It is not fully specified. 15) org.jclouds.rest.RestContext<org.jclouds.aws.s3.AWSS3Client, A> cannot be used as a key; It is not fully specified. 16) No implementation for org.jclouds.rest.HttpClient was bound. at org.jclouds.rest.config.BinderUtils.bindHttpApiProvider(BinderUtils.java:107) 17) No implementation for org.jclouds.aws.s3.AWSS3Client was bound. at org.jclouds.rest.config.BinderUtils.bindHttpApiProvider(BinderUtils.java:107) 17 errors com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435) com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154) com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106) com.google.inject.Guice.createInjector(Guice.java:95) org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:405) org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:329) org.jclouds.ContextBuilder.buildView(ContextBuilder.java:620) org.jclouds.ContextBuilder.buildView(ContextBuilder.java:600) org.apache.usergrid.services.assets.data.S3BinaryStore.getContext(S3BinaryStore.java:87) org.apache.usergrid.services.assets.data.S3BinaryStore.read(S3BinaryStore.java:190) org.apache.usergrid.services.assets.data.S3BinaryStore.read(S3BinaryStore.java:208) org.apache.usergrid.rest.applications.ServiceResource.executeStreamGet(ServiceResource.java:642) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) c...3
200USERGRID-580Docs for new Central Usergrid SSO featureThis should be documented, it's a nice new feature for Usergrid 1.0.2.2
201USERGRID-581Implement ability to createItemInCollection using the graph.This should fix the broken test in the EntityManagerIt labeled: ownershipScopeCorrect3
202USERGRID-582Test Not Passing: CollectionIT TestsNULL3
203USERGRID-584Test Not Passing: staleIndexCleanupSeems like all of the stale entities aren't being cleaned up. 1
204USERGRID-585Test migration from old index format to newTest migration from old index format to new3
205USERGRID-586Test app_info migrationNULL3
206USERGRID-587Test Not Passing: IteratingQueryIT + collectionITNULL1
207USERGRID-588Test and confirm Raw Push Notifications on WindowsNULL3
208USERGRID-589Complete SNS/SQS async indexing apiPublish to SNS Consume from SQS Complete the SQS indexing impl with the new message endpoint add new interface for sqs vs in memory service 3
209USERGRID-592[SPIKE] Investigate support for Central Usergrid SSO in the PortalIf Central Usergrid then the Portal should redirect the user to the central Usergrid instance for login/registration. And, when the Portal gets an Admin User token from the Central Usergrid SSO system, the Portal must validate it against the Usergrid system with which it is configured to work. This is a spike to design a possible solution for SSO support in portal.3
210USERGRID-593Implement Read by Graph Edges for no QueryWhen no query is specified, we should read by graph edges, and not read via elasticsearch. The low level seek logic exists. The relationManager simply needs to wire this functionality in. A new Graph I/O interface should be created to cleanly implement and test this.5
211USERGRID-595Hystrix/Cassandra exception falsely reportedAPI response of HystrixCassandra error is being reported when ElasticSearch takes too long (not cassandra): {"error":"hystrix_runtime","timestamp":1429979394729,"duration":0,"error_description":"HystrixCassandra$1 timed-out and no fallback available.","exception":"com.netflix.hystrix.exception.HystrixRuntimeException"} This can be reproduced by attaching a debugger and pausing in EsEntityIndexBatchImpl.execute3
212USERGRID-596Confirm PUT by Name Does not Fetch/Query ESWe are seeing a large number of fetches/queries when we are doing PUT by name. This creates load on ES and therefore makes indexing take longer. We should make sure we're not hitting ES for PUT by name.1
213USERGRID-598Merge two-dot-o-dev into two-dot-oNULL3
214USERGRID-599Cannot get events after they have been postedAfter posting an event to the event collection, there is no way to get the individual event back out, or query for the event. Attatched is a sample of creating an event in "sandbox", and immediately trying to retrieve it: Alans-MacBook-Air:~ ApigeeCorporation$curl -X POST https://api.usergrid.com/twyder/sandbox/events -d '{"timestamp": 0, "counters" : {"mycounter_20":1}}' { "action" : "post", "application" : "93554305-3e6a-11e2-9141-02e81ae640dc", "params" : { }, "path" : "/events", "uri" : "https://api.usergrid.com/twyder/sandbox/events", "entities" : [ { "uuid" : "c06510a0-83a4-182e-ba26-7f45da741a63", "type" : "event", "created" : 1430241847214, "modified" : 1430241847214, "timestamp" : 1430241847214, "counters" : { "mycounter_20" : 1 }, "message" : null, "metadata" : { "path" : "/events/c06510a0-83a4-182e-ba26-7f45da741a63" } } ], "timestamp" : 1430241847212, "duration" : 16, "organization" : "twyder", "applicationName" : "sandbox" }Alans-MacBook-Air:~ ApigeeCorporation$ curl -X GET https://api.usergrid.com/twyder/sandbox/events/c06510a0-83a4-182e-ba26-7f45da741a63 {"error":"service_resource_not_found","timestamp":1430241876216,"duration":0,"exception":"org.apache.usergrid.services.exceptions.ServiceResourceNotFAlans-MacBook-AAlans-MacBooAlAlA3
215USERGRID-601Test Usergrid Central SSONULL3
216USERGRID-602Test Import / Export of Admin UsersNULL2
217USERGRID-604Spike: Core test suite should complete in MavenNULL1
218USERGRID-606Implement Async Indexing on New EdgeNULL1
219USERGRID-607Implement Deleting Edge out of indexquerying es for all edges deleting in bulk async using async index service 1
220USERGRID-608Refactor/Complete Delete Entity in 2.1can o worms refactor cp relation manager into async service async == async event service starts process look at indexingservice 2
221USERGRID-609Document the new IO frameworkNULL2
222USERGRID-612Test Not Passing: EntityConnectionsITNULL3
223USERGRID-613Reindex endpoint should not be case-sensitivethe reindex endpoint: http://localhost:8080/system/index/rebuild/a34ad389-b626-11e4-848f-06b49118d7d0/Storelatlons does not work due to case sensitivity.1
224USERGRID-614Refactor compaction operations to be declarative in 2.1Currently our event system originates from too may points. Both GraphManager and CollectionManager fire events for compaction. This is causing issues with our event orchestration when we aggregate events in the core tier. A quick refactor needs to occur. The following will need to happen. 1) Define new compaction interfaces on the EntityCollectionManager and GraphManager 2) Refactor the existing code to be invoked on these events 3) Invoke the compaction at a higher level. Mark will sill occur, compaction will be deferred to the runtime. This allows a separate distributed subsystem to be created. 3
225USERGRID-616Queries using greater/less than on float fields failNULL3
226USERGRID-617Queries using order by on float fields result in incorrect ordering NULL3
227USERGRID-618Additional testing of Usergrid SSONULL3
228USERGRID-621Test Not Passing: InMemoryAsyncIndexServiceTestNULL2
229USERGRID-622Test Not Passing: IndexServiceTestNULL2
230USERGRID-623Test Not Passing: AllEntitiesInSystemObservableITNULL2
231USERGRID-624Test Not Passing: MessagesITNULL2
232USERGRID-625Test Not Passing: CountingMutatorITNULL2
233USERGRID-626Test Not Passing: EntityDictionaryITNULL2
234USERGRID-628Test Not Passing: GeoITNULL1
235USERGRID-629Test Not Passing: GeoQueryBooleanTestNULL2
236USERGRID-630Test Not Passing: IndexITNULL2
237USERGRID-631Test Not Passing: PathQueryITNULL2
238USERGRID-632Test Not Passing: PerformanceEntityRebuildIndexTestNULL1
239USERGRID-634Test Not Passing: EntityManagerFactoryImplITNULL1
240USERGRID-635Test Not Passing: IntersectionTransitivePagingITNULL2
241USERGRID-636Test Not Passing: IntersectionUnionPagingITNULL2
242USERGRID-637Test Not Passing: NotSubPropertyITNULL2
243USERGRID-639Entity does not return from collection when ql=select *Entity exists at /{org}/{app}/{collection}/{name} but is not returned from /{org}/{app}/{collection}?ql=select * For reference: https://apigeesc.atlassian.net/browse/APIBAAS-15603
244USERGRID-640Entities created on the portal during 'create collection' do not get persistedIn the portal, when you use the create collection button to create a collection you now must specify the first entity on the form. It does not appear that the entity gets persisted into the collection. This can be recreated on the Usergrid free version at http://appservices.apigee.com/index.html/#!/login3
245USERGRID-641Implement search by connections in core EntityConnectionsIT.testEntityConnectionsMembership:271 null EntityConnectionsIT.testEntityConnectionsSimple:79 null are failing3
246USERGRID-642Test Not Passing: StaleIndexCleanup Failing due to old version getting returned from searchjava.lang.AssertionError: Expected :e2283e4d-f504-11e4-b4ae-324ce75ff58b Actual :e2150469-f504-11e4-b4ae-324ce75ff58b <Click to see difference> org.junit.Assert.fail(Assert.java:88) org.junit.Assert.failNotEquals(Assert.java:834) org.junit.Assert.assertEquals(Assert.java:118) org.junit.Assert.assertEquals(Assert.java:144) org.apache.usergrid.corepersistence.StaleIndexCleanupTest.testUpdateVersionMaxFirst(StaleIndexCleanupTest.java:187) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.usergrid.CoreApplication$1.evaluate(CoreApplication.java:145) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) org.apache.usergrid.CoreITSetupImpl$1.evaluate(CoreITSetupImpl.java:76) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.run(ParentRunner.java:363) org.junit.runner.JUnitCore.run(JUnitCore.java:137) com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78) com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212) com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) 3
247USERGRID-643Implement Reindex By CollectionPerformanceEntityRebuildIndexTest.rebuildOneCollectionIndex:228 null3
248USERGRID-644Test Not Passing: IteratingQueryIt failing due to bad serializertrying to deserialize a markededge into an edge AbstractCursorSerializer line 47 2015-05-08 14:33:25,926 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205920001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a69ba0a-f5c1-11e4-ab26-47475f9887d5, type='test'}, version=7a69ba63-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:25,942 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205934001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a6bdcea-f5c1-11e4-8198-83878cd74a93, type='test'}, version=7a6c0456-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:25,957 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205949001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a6e26da-f5c1-11e4-8f8d-9b115b1298d8, type='test'}, version=7a6e2739-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:25,970 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205964001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a7070ca-f5c1-11e4-9deb-7d7bf418ca86, type='test'}, version=7a70712c-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:25,984 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205978001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a7293aa-f5c1-11e4-8cbc-6b63f058386d, type='test'}, version=7a72bb1f-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:25,998 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117205991001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a748f7a-f5c1-11e4-b755-cfcc9cf90313, type='test'}, version=7a748fe2-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:26,013 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117206006001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a76d96a-f5c1-11e4-b6ba-5103774b6f21, type='test'}, version=7a7700e5-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:26,027 DEBUG (main) IndexServiceImpl - adding edge IndexEdgeImpl{timestamp=1431117206021001} SearchEdgeImpl{nodeId=SimpleId{uuid=7a1f1c82-f5c1-11e4-9ef9-324ce75ff58b, type='application'}, name='zzzcollzzz|tests', nodeType=TARGET} to batch for entity Entity{id=SimpleId{uuid=7a79235a-f5c1-11e4-a640-1f069a6b43c9, type='test'}, version=7a7923c8-f5c1-11e4-9ef9-324ce75ff58b} 2015-05-08 14:33:26,112 INFO (main) IndexRefreshCommandImpl - found record during refresh uuid: 7a7c0980-f5c1-11e4-b2bd-5994708e0639 took ms:75 2015-05-08 14:33:26,112 INFO (main) IteratingQueryIT - Writes took 571 ms Disconnected from the target VM, address: '127.0.0.1:49588', transport: 'socket' 2015-05-08 14:34:59,660 INFO (main) CoreApplication - Test allInConnectionNoType(org.apache.usergrid.persistence.query.IteratingQueryIT): finish with application org.apache.usergrid.corepersistence.pipeline.cursor.CursorParseException: Unable to deserialize value org.apache.usergrid.corepersistence.pipeline.cursor.AbstractCursorSerializer.fromJsonNode(AbstractCursorSerializer.java:51) org.apache.usergrid.corepersistence.pipeline.cursor.RequestCursor.getCursor(RequestCursor.java:75) org.apache.usergrid.corepersistence.pipeline.PipelineContext.getCursor(PipelineContext.java:68) org.apache.usergrid.corepersistence.pipeline.read.AbstractPathFilter.getSeekValue(AbstractPathFilter.java:50) org.apache.usergrid.corepersistence.pipeline.read.graph.AbstractReadGraphFilter.lambda$call$2(AbstractReadGraphFilter.java:73) org.apache.usergrid.corepersistence.pipeline.read.graph.AbstractReadGraphFilter$$Lambda$100/1957269967.call(Unknown Source) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:43) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:32) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.subscribe(Observable.java:7585) rx.internal.operators.BlockingOperatorToIterator.toIterator(BlockingOperatorToIterator.java:53) rx.observables.BlockingObservable.getIterator(BlockingObservable.java:156) org.apache.usergrid.corepersistence.results.ObservableQueryExecutor.hasNext(ObservableQueryExecutor.java:114) org.apache.usergrid.corepersistence.results.ObservableQueryExecutor.next(ObservableQueryExecutor.java:124) org.apache.usergrid.corepersistence.CpRelationManager.searchConnectedEntities(CpRelationManager.java:948) org.apache.usergrid.corepersistence.CpEntityManager.searchConnectedEntities(CpEntityManager.java:1546) org.apache.usergrid.persistence.query.IteratingQueryIT$ConnectionNoTypeHelper.getResults(IteratingQueryIT.java:278) org.apache.usergrid.persistence.query.IteratingQueryIT.allIn(IteratingQueryIT.java:1130) org.apache.usergrid.persistence.query.IteratingQueryIT.allInConnectionNoType(IteratingQueryIT.java:71) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) org.apache.usergrid.CoreApplication$1.evaluate(CoreApplication.java:145) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) org.apache.usergrid.CoreITSetupImpl$1.evaluate(CoreITSetupImpl.java:76) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.run(ParentRunner.java:363) org.junit.runner.JUnitCore.run(JUnitCore.java:137) com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68) Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "deleted" (class org.apache.usergrid.persistence.graph.impl.SimpleEdge), not marked as ignorable (4 known properties: "type", "targetNode", "sourceNode", "timestamp"]) at [Source: N/A; line: -1, column: -1] (through reference chain: org.apache.usergrid.persistence.graph.impl.SimpleEdge["deleted"]) com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51) com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:671) com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:773) com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1297) com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1275) com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:247) com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118) com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2965) com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1587) com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:1931) org.apache.usergrid.corepersistence.pipeline.cursor.AbstractCursorSerializer.fromJsonNode(AbstractCursorSerializer.java:48) ... 74 more Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: org.apache.usergrid.corepersistence.pipeline.read.FilterResult.class rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:101) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:58) ... 68 more2
249USERGRID-645Update superuser to allow all system accessCurrently superuser access does not allow access to all resources, as a superuser this should be allowed. If SuperUser is evaluated as a Role, this may be a more optimal solution from a management perspective.3
250USERGRID-647Issue with Unique Values - NPEPlease ensure appropriate null checks and behavior happen here: 2015-05-12 17:05:37,811 [http-bio-8080-exec-14] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- java.lang.NullPointerException Server Error (500) java.lang.NullPointerException org.apache.usergrid.services.AbstractConnectionsService.getItemByName(AbstractConnectionsService.java:238) org.apache.usergrid.services.AbstractService.invokeItemWithName(AbstractService.java:671) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:628) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.invokeMultiple(ServiceRequest.java:262) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:229) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executeGet(ServiceResource.java:297) sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:497) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745)3
251USERGRID-652Index Routing/Location StrategyWe should consider implementing a proxy which provides the index being used for an application. This proxy should provide both the Index strategy and the entity type mapping strategy. The goal is to provide a point where we can implement a strategy function that could do one (or more) of the following strategies for indexes: - Single index per cluster (static index allocation) - Index per org/app (dynamic index allocation) - Index per org (dynamic index allocation) - Index per cluster (dynamic index allocation) - Index per customer (dynamic index allocation) - Hash/bucketing across a fixed range of indexes (static index allocation) - Per environment (TBD) - Static (for the management app, should always match the Cassandra keyspace name as the index) In order to do this we need to update EntityIndexFactory.createApplicationEntityIndex. For Entity Type Mapping, the goal is to enable bucketing of entity types (collections) in a manner that can be more optimal for the indexing strategy that is chosen. For example, a single index strategy might have a single entity type but for the index per org/app it might have a separate entity type. Looking up the management index would have a strategy where the index name matches the cassandra keyspace name5
252USERGRID-654AwsSdkS3BinaryStore dependencies break the REST module testsThe new AwsSdkS3BinaryStore implementation depends on Jackson 2.3.x data mapper, which conflicts with Jackson 1.x and causes the REST tests to fail. If you use Maven to exclude the Jackson 2.3.x object mapper (which I did with PR https://github.com/apache/incubator-usergrid/pull/244) and configure the REST tests to use AwsSdkS3BinaryStore, then the REST tests fail with the exception below, which indicates to me that AwsSdkS3BinaryStore will not work at runtime without Jackson 2.3.x object mapper. /cc [~pandacat]: do you have any ideas about how to work around this issue? Have you had success with the REST module tests and your new AWS SDK based BinaryStore implementation? java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper at com.amazonaws.util.json.Jackson.<clinit>(Jackson.java:32) at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:230) at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:247) at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:282) at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:139) at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:134) at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:95) at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:42) at org.apache.usergrid.services.assets.data.AwsSdkS3BinaryStore.getS3Client(AwsSdkS3BinaryStore.java:98) 3
253USERGRID-655More than one connection does not get created when running batch index on connection edgesWhen running the following code in indexEntity final Observable<IndexOperationMessage> batches = observable.buffer( indexFig.getIndexBatchSize() ) //map into batches based on our buffer size .flatMap( buffer -> Observable.from( buffer ) //collect results into a single batch .collect( () -> ei.createBatch(), ( batch, indexEdge ) -> { logger.debug( "adding edge {} to batch for entity {}", indexEdge, entity ); batch.index( indexEdge, entity ); } ) //return the future from the batch execution .flatMap( batch -> batch.execute() ) ); When run with more than one SearchEdge only one SearchEdge is indexed into elasticsearch. Despite both search edges having the same document id. The batch executes but only one connection gets indexed in elasticsearch. 3
254USERGRID-657Graph deletes not going through. This call returns an empty iterator storageEdgeSerialization.getEdgeVersions( scope, new SimpleSearchByEdge( edge.getSourceNode(), edge.getType(), edge.getTargetNode(), Long.MAX_VALUE, SearchByEdgeType.Order.ASCENDING, Optional.absent() ) ); when we go to delete the edge. So since its empty it returns null and fails the test. final Edge toBeDeletedEdge = graphManager.deleteEdge( connectionSearch ).toBlocking().firstOrDefault( null ); Need to resolve this so we can fix graph tests. 3
255USERGRID-659Test Not Passing: OrganizationIT FailingNULL3
256USERGRID-660Test Not Passing: GroupServiceIT - ServicesNULL1
257USERGRID-661Test Not Passing: ServiceInvocationITNULL1
258USERGRID-662Test Not Passing: AppInfoMigrationPluginTestNULL1
259USERGRID-663Test Not Passing: ConnectionsServiceITNULL1
260USERGRID-664Test Not Passing: RolesServiceIT.deleteRolesNULL1
261USERGRID-6661.0.2 Release Candidate 1NULL2
262USERGRID-6671.0.2 Release Candidate 2NULL2
263USERGRID-669Update edge and type naming conventionsNULL1
264USERGRID-670App Credentials do not work when accessing a collection within an app (2.0)In order to access collections it seems the org credentials are required as app credentials do not work. 5
265USERGRID-671401 ServiceResourceNotFoundException instead of expected 404When you do a get by name/uuid of an entity which does not exist a 401 error is returned instead of a 404.2
266USERGRID-672Portal stores the password of an app user after it is created in clear-textAfter creating an app user their password is stored in clear-text in the portal. It should not be persisted in the session or displayed in the portal in clear-text IMPORTANT NOTE: the current version of the Portal that we use (at Apigee) is setup to enable the new Usergrid Central SSO, so if it is to be deployed before Central SSO is enabled in prod, then you will have to disable central SSO in the portal before deploying it.3
267USERGRID-675Refactor SearchRequestBuilder to create more dynamic queries.Currently using the SearchRequestBuilder add on some custom fields. This was fine when we just needed to do a "one size fits all " to our search method but now that there are more methods being introduced we need a better way to create queries that are efficient for the type of processing that is going to be done. 5
268USERGRID-676Update mapping to use "dynamic": "strict" to prevent new fields from being added dynamically"dynamic": "strict" We should ensure this is not used per https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-dynamic-mapping.html1
269USERGRID-677Create new parser for Scrolling apiCurrently we only do search results and parse those results under the assumption that we will only do one search. As we query elasticsearch we want to be able to use the scrolling api to batch pull all the results we can find from elasticsearch. 1
270USERGRID-678ElasticSearch Type Mapping has _all in wrong place_all should be moved to a child of 'entity'1
271USERGRID-679Update SQS provider implementation to use ObservableSQS Queue Manager should be updated to use Observable and emit a stream of messages.2
272USERGRID-680Merge master -> two-dot-o: SSO & Assets changesNULL1
273USERGRID-682Soak Testing ScriptsCreate scripts that will run for a period of time and generate data/load. Start scripts and run for a week.3
274USERGRID-684UG 1.0 bug for DB setup with separate locks keyspaceIt seems the keyspace does not get created with /database/setup. cassandra.system.keyspace=sso_ug10 #cassandra.application.keyspace=Usergrid_Applications cassandra.application.keyspace=sso_ug10_apps #cassandra.lock.keyspace=Locks cassandra.lock.keyspace=sso_ug10_Locks3
275USERGRID-685Test Not Passing: ConnectionsServiceIT.testEntityConnectionsNULL3
276USERGRID-686Test Not Passing: PagingResourceITthere are two test failures in intelliJ testCursor: org.junit.ComparisonFailure: Expected :1 Actual :11 <Click to see difference> org.junit.Assert.assertEquals(Assert.java:115) org.junit.Assert.assertEquals(Assert.java:144) org.apache.usergrid.rest.applications.collection.paging.PagingResourceIT.pageAndVerifyEntities(PagingResourceIT.java:305) org.apache.usergrid.rest.applications.collection.paging.PagingResourceIT.testCursor(PagingResourceIT.java:149) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.usergrid.rest.test.resource2point0.ClientSetup$1.evaluate(ClientSetup.java:82) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) org.junit.runners.ParentRunner.run(ParentRunner.java:363) org.junit.runner.JUnitCore.run(JUnitCore.java:137) com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78) com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212) com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) and pagingEntities org.junit.ComparisonFailure: Expected :1 Actual :100 <Click to see difference> org.junit.Assert.assertEquals(Assert.java:115) org.junit.Assert.assertEquals(Assert.java:144) org.apache.usergrid.rest.applications.collection.paging.PagingResourceIT.pageAndVerifyEntities(PagingResourceIT.java:305) org.apache.usergrid.rest.applications.collection.paging.PagingResourceIT.pagingEntities(PagingResourceIT.java:177) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) org.apache.usergrid.rest.test.resource2point0.ClientSetup$1.evaluate(ClientSetup.java:82) org.junit.rules.RunRules.evaluate(RunRules.java:20) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) org.junit.runners.ParentRunner.run(ParentRunner.java:363) org.junit.runner.JUnitCore.run(JUnitCore.java:137) com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78) com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212) com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)2
277USERGRID-687Test Not Passing: AndOrQueryTestFix AndOrQueryTest: queryReturnCheck queryReturnCheckWithShortHand1
278USERGRID-688Fix Graph Compact Operations. when mark node in graph is called the compact needs to be called before you add the same relationship back to the graph. this should not be the case5
279USERGRID-689Test Not Passing: ManagementResourceIT11 tests failing2
280USERGRID-690Test Not Passing: OrganizationsITNULL2
281USERGRID-692Test Not Passing: PermissionsResourceIT PermissionsResourceIT.applicationPermissions:262 expected:<[noca]> but was:<[4peaks]> PermissionsResourceIT.deleteUserGroup:157 null2
282USERGRID-693Test Not Passing: ContentTypeResourceITContentTypeResourceIT.formEncodedContentType:152 expected:<200> but was:<500> ContentTypeResourceIT.noAcceptGet:262 expected:<200> but was:<400>2
283USERGRID-695Test Not Passing: RegistrationIT RegistrationIT.addExistingAdminUserToOrganization:304 » UniformInterface POST ... RegistrationIT.addNewAdminUserWithNoPwdToOrganization:245->postAddAdminToOrg:90 » ClientHandler RegistrationIT.postAddToOrganization:222->AbstractRestIT.getAdminToken:173 » UniformInterface RegistrationIT.postCreateOrgAndAdmin:134->AbstractRestIT.getAdminToken:173 » UniformInterface RegistrationIT.putAddToOrganizationFail:195->AbstractRestIT.getAdminToken:173 » UniformInterface2
284USERGRID-696AdminEmailEncodingIT AdminEmailEncodingIT.getTokenDash:73->doTest:116 » UniformInterface POST http:... AdminEmailEncodingIT.getTokenPlus:53->doTest:116 » UniformInterface POST http:... AdminEmailEncodingIT.getTokenUnderscore:63->doTest:116 » UniformInterface POST...2
285USERGRID-697Test Not Passing: AdminUsersITAdminUsersIT.mgmtUserFeed:190 » UniformInterface GET http://localhost:10006/ma...2
286USERGRID-698Test Not Passing: ConnectionResourceTest.connectionsLoopbackTest:103 NullPointerNULL2
287USERGRID-699Test Not Passing: NotificationsIT.testPaging:90NULL2
288USERGRID-700Test Not Passing: StaleIndexCleanupTestsResults : Failed tests: StaleIndexCleanupTest.testCleanupOnUpdate:462 Expect candidates without earlier stale entities expected:<10> but was:<60> Tests run: 258, Failures: 1, Errors: 0, Skipped: 251
289USERGRID-701Rewrite Registration tests to fully cover the registration flow.The current set of tests have very few comments and don't logically make sense in some cases. We need more tests to expose the registration flow and possible edge cases when certain properties aren't correctly set. 3
290USERGRID-702Rest tests return 403 when run in maven, but pass in intelliJSo I'm getting responses like the following: com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:10003/management/token returned a response status of 403 Forbidden com.sun.jersey.api.client.WebResource.handle(WebResource.java:688) com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) org.apache.usergrid.rest.test.resource2point0.endpoints.NamedResource.post(NamedResource.java:192) org.apache.usergrid.rest.test.resource2point0.AbstractRestIT.getAdminToken(AbstractRestIT.java:167) org.apache.usergrid.rest.management.OrganizationsIT.testCreateOrgUserAndReturnCorrectUsername(OrganizationsIT.java:332) testCreateDuplicateOrgEmail(org.apache.usergrid.rest.management.OrganizationsIT) Time elapsed: 0.988 sec <<< ERROR! com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:10003/management/token returned a response status of 403 Forbidden com.sun.jersey.api.client.WebResource.handle(WebResource.java:688) com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) org.apache.usergrid.rest.test.resource2point0.endpoints.NamedResource.post(NamedResource.java:192) org.apache.usergrid.rest.management.OrganizationsIT.testCreateDuplicateOrgEmail(OrganizationsIT.java:172) testOrgPOSTForm(org.apache.usergrid.rest.management.OrganizationsIT) Time elapsed: 0.819 sec <<< ERROR! com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:10003/management/token returned a response status of 403 Forbidden com.sun.jersey.api.client.WebResource.handle(WebResource.java:688) com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) org.apache.usergrid.rest.test.resource2point0.endpoints.NamedResource.post(NamedResource.java:192) org.apache.usergrid.rest.management.OrganizationsIT.testOrgPOSTForm(OrganizationsIT.java:277) createOrgAndOwner(org.apache.usergrid.rest.management.OrganizationsIT) Time elapsed: 0.774 sec <<< ERROR! com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:10003/management/token returned a response status of 403 Forbidden com.sun.jersey.api.client.WebResource.handle(WebResource.java:688) com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) org.apache.usergrid.rest.test.resource2point0.endpoints.NamedResource.post(NamedResource.java:192) org.apache.usergrid.rest.management.OrganizationsIT.createOrgAndOwner(OrganizationsIT.java:80) testCreateDuplicateOrgName(org.apache.usergrid.rest.management.OrganizationsIT) Time elapsed: 0.975 sec <<< ERROR! com.sun.jersey.api.client.UniformInterfaceException: POST http://localhost:10003/management/token returned a response status of 403 Forbidden com.sun.jersey.api.client.WebResource.handle(WebResource.java:688) com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) org.apache.usergrid.rest.test.resource2point0.endpoints.NamedResource.post(NamedResource.java:192) org.apache.usergrid.rest.management.OrganizationsIT.testCreateDuplicateOrgName(OrganizationsIT.java:118) So Calls to management/token are failing with 403 errors. I suspect it might be due to the new testing framework but further investigation is needed5
291USERGRID-703Test Not Passing: ManagementResourceITtestSuperuserOnlyWhenValidateExternalTokensEnabled and the other external test are broken.2
292USERGRID-704Test Not Passing: DevicesresourceIT seems to be a graph compaction issue DevicesResourceIT.putWithUUIDShouldCreateAfterDelete1
293USERGRID-705Add organizations to Admin User import/export (usergrid-tools.jar)TBD3
294USERGRID-706Test re-index systemThe first pass of the re-index with is complete. Testing in a system to ensure that re-index works as expected still needs to be completed.5
295USERGRID-707ApplicationResource can no longer update user token ttl's. ApplicationResourceIT.updateAccessTokenTtl() is broken because of the added funcitonality of app delete. The way it was implemented added a executePut method into the ApplicationResource which overrides the service resource executePut method thus breaking this test. What we want to decide moving forward is up to the person who works on this ticket. 2
296USERGRID-708cursor in username field on login jumps to the end with updateNULL1
297USERGRID-709Notifications disappear too quickly. Increase timeout and have them stay on mouseoverNULL2
298USERGRID-710Create an area where all notifications for a session are viewableNULL3
299USERGRID-711Allow the ability to view more that 10 applications in the applications dropdown and on the organization administration viewNULL2
300USERGRID-713Don't modify the path when an entity is selected in the data viewNULL2
301USERGRID-714Reorganize the query area on data view so that more of the data is visibleNULL2
302USERGRID-715Decrease line spacing/padding so that more data is visible in list viewsNULL1
303USERGRID-716Use combo boxes instead of text areas for inputting multiple pieces of dataNULL3
304USERGRID-719Pushy creates/leaves too many threads Pushy client opens up hundreds of threads and has some failed connection alerts which need to be investigated.3
305USERGRID-720Reconcile default property settings with known-good production settingsSettings which are used in a production environment should be the default in the Figs. This will simplify the configuration management required when running Usergrid at scale.2
306USERGRID-724JMX Metrics/Counters for CassandraWould be nice to add metrics and counters at the lower level for bytes in/out Cassandra and latency1
307USERGRID-725SNSQueueManagerImpl needs better error handling and messagingIt would be better if this was preceded by a message stating which queue cannot be found. com.amazonaws.services.sqs.model.QueueDoesNotExistException: The specified queue does not exist for this wsdl version. (Service: AmazonSQS; Status Code: 400; Error Code: AWS.SimpleQueueService.NonExistentQueue; Request ID: 70c94af9-7d46-5d2b-9a7e-07155ddd1ed5) com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160) com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748) com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467) com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302) com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2422) com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1130) org.apache.usergrid.persistence.queue.impl.SNSQueueManagerImpl.getMessages(SNSQueueManagerImpl.java:234) org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService.take(AmazonAsyncEventService.java:153) org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService.access$300(AmazonAsyncEventService.java:66) org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService$2.call(AmazonAsyncEventService.java:379) org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService$2.call(AmazonAsyncEventService.java:366) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62) rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745)1
308USERGRID-726Audit and review JMX Metrics for consistencyJMX Metrics have varying names and formats. It would be helpful to standardize them. Lets take a pass and rename them to have an "x.y" pattern. Where x=object/concept and y=attribute/action. If there is no explicit y, y should be 'base'. Here is an example: this.deleteApplicationMeter = metricsFactory.getMeter(EsApplicationEntityIndexImpl.class, "application.delete"); https://github.com/apache/incubator-usergrid/pull/2692
309USERGRID-727Issue with getting started steps on githubThe getting started documents points to some Apigee links and hence don't provide the expected results. This could drive away potential users. Documentation needs to be updated. link: https://github.com/apache/incubator-usergrid section: Getting Started with the Admin Portal Unable to access: - https://github.com/apigee/usergrid-portal - http://apigee.github.com/usergrid-portal/?api_url=http://localhost:8080 Rgds, Malaka3
310USERGRID-728Launcher url points to ApigeeThe launcher contains the url to: http://apigee.github.io/usergrid-portal/. This should be changed to reflect the new url. Do we still host an admin portal on github like Apigee used to? ... which we can point out local instance to by specifying an api_url argument in the query string? Rgds, Malaka3
311USERGRID-729SNS Queue Manager uses ARN to read instead of URLThere is a bug in org.apache.usergrid.persistence.queue.impl.SNSQueueManagerImpl which results in the ARN of the queue being used when the URL should be used.1
312USERGRID-732Implement SNS-based Queue Provider in 2.0NULL2
313USERGRID-734Rationalize JMX Metrics in 2.0 branchMake same as 2.1 branch2
314USERGRID-7382.0 does not correctly handle inferred type of Date with optional timeThis causes issues if the field is blank. We need to resolve this for a production installation.3
315USERGRID-739Implement Bucketing Algorithm for assignment of Application Index Aliases to Physical IndexesWhen allocating an index alias to one of the physical indexes a bucketing strategy should be used with ranges based off the number of indexes in elasticsearch. For example, if there are 10 physical indexes then there would be 10 buckets based on a hash of the Application ID which would result in the assignment of an app alias to a physical index based on the range in which it falls. 3
316USERGRID-740Temp files from asset upload not removed, causing the file system to use all inodesAfter running Usergrid 2.0 for over 1 month on a tomcat, we have saturated the inodes of the disk on the tomcat machine. Our temporary files for uploads are not getting removed. Files of this structure appear in {code} ls -lrt /var/cache/tomcat7/temp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME377627531473294092.tmp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME2634951521985073318.tmp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME1066361445384372180.tmp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME3554741621747313255.tmp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME1805372544199883785.tmp -rw-r--r-- 1 tomcat tomcat 0 Jun 9 23:23 MIME414525331725926400.tmp {code} We need to ensure that all temporary files are removed. Note that all the files remaining are 0 bytes. This seems to be caused by an edge case when a 0 byte file is uploaded, or possibly the upload is failing. 5
317USERGRID-744Test Not Passing: ApplicationResourceITNULL2
318USERGRID-745Test Not Passing: AssetResourceITNULL1
319USERGRID-746Improve performance of ExportAdmins via multi-threadingNULL3
320USERGRID-748SNS Queueing/Messaging should create RAW subscriptionsSNS wraps messages passed to SQS in an envelope unless you specify a RAW subscription type. At this time we would prefer a RAW message delivery.1
321USERGRID-749NPE running (resuming?) reindex with no payload in the requestThe following occurs when reindexing the system after indexes are deleted. It should recreate the indexes if they do not exist. 2015-06-12 00:25:24,227 [http-bio-8080-exec-14] INFO org.apache.usergrid.rest.security.shiro.filters.BasicAuthSecurityFilter- System administrator access allowed 2015-06-12 00:25:24,253 [http-bio-8080-exec-14] INFO org.apache.usergrid.rest.IndexResource- Resuming rebuilding all applications 2015-06-12 00:25:24,256 [http-bio-8080-exec-14] ERROR org.apache.usergrid.rest.exceptions.ThrowableMapper- An uncaught exception occurred during HTTP invocation java.lang.NullPointerException org.apache.usergrid.rest.IndexResource.executeResumeAndCreateResponse(IndexResource.java:274) org.apache.usergrid.rest.IndexResource.rebuildIndexesPut(IndexResource.java:102) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:483) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) 2015-06-12 00:25:24,257 [http-bio-8080-exec-14] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- org.apache.usergrid.rest.exceptions.UncaughtException Server Error (500) org.apache.usergrid.rest.exceptions.UncaughtException: java.lang.NullPointerException org.apache.usergrid.rest.exceptions.AbstractExceptionMapper.toResponse(AbstractExceptionMapper.java:59) org.apache.usergrid.rest.exceptions.ThrowableMapper.toResponse(ThrowableMapper.java:37) com.sun.jersey.spi.container.ContainerResponse.mapException(ContainerResponse.java:480) com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException(ContainerResponse.java:417) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1477) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException org.apache.usergrid.rest.IndexResource.executeResumeAndCreateResponse(IndexResource.java:274) org.apache.usergrid.rest.IndexResource.rebuildIndexesPut(IndexResource.java:102) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:483) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) ... 38 more 2015-06-12 00:25:24,258 [http-bio-8080-exec-14] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- Uncaught Exception org.apache.usergrid.rest.exceptions.UncaughtException: java.lang.NullPointerException org.apache.usergrid.rest.exceptions.AbstractExceptionMapper.toResponse(AbstractExceptionMapper.java:59) org.apache.usergrid.rest.exceptions.ThrowableMapper.toResponse(ThrowableMapper.java:37) com.sun.jersey.spi.container.ContainerResponse.mapException(ContainerResponse.java:480) com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException(ContainerResponse.java:417) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1477) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException org.apache.usergrid.rest.IndexResource.executeResumeAndCreateResponse(IndexResource.java:274) org.apache.usergrid.rest.IndexResource.rebuildIndexesPut(IndexResource.java:102) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:483) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) ... 38 more 2015-06-12 00:25:24,271 [http-bio-8080-exec-14] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- Server Error (500): {"error":"uncaught","timestamp":1434068724258,"duration":13,"error_description":"Internal Server Error","exception":"org.apache.usergrid.rest.exceptions.UncaughtException","error_id":"846ee932-1099-11e5-a3f5-06b0ee5388b2"}3
322USERGRID-750When setting accesstokenttl to 0, client credentials auth still returns 604800If I send a PUT request to set the accesstokenttl to 0 for an app: {code} { "accesstokenttl": 0 } {code} And then make a subsequent client_credentials token request, the token ttl in the response is still 604800: {code} { "client_id": "***", "client_secret": "***", "grant_type": "client_credentials", "ttl": 0 } {code} Response: {code} { "access_token": "***", "expires_in": 604800, "application": "***" } {code} [~tnine] suggests this may be a response rendering problem and that it is actually setting the expiration correctly in the stack.3
323USERGRID-751Change MvccEntitySerializationStrategyV3Impl to use native JSON when storing in the column valueCurrently we use SMILE binary serialization to serialize our column values. During production debugging, this makes it extremely difficult to debug. I propose we remove the SMILE binary serialization, and instead use json serialization with the following formation {code} { "version": [serialization implementation version number as an int], "entity": {... entity as a nested json object } } {code} If the entity is present, then the entity has not been deleted. If the entity field is not present, then the entity should be read as a marked for deletion. 3
324USERGRID-752Update Usergrid 1.0 search to use concurrent search per shardCurrently we merge all shards together with every page seek in usergrid 1.0. This means we select 20k columns, and discard 19k with each page selection. Instead, I propose that we create an AST per shard, and execute the iterator on these shards concurrently. We then merge on the last step, greatly reducing the amount of network I/O and discarded data on queries of large data sets. Proposed Changes: 1 AST instance per shard, evaluated concurrently via some worker pool. Change final gather of concurrent iterators. Change cursor generation to create the cursor from the merged trees. 3
325USERGRID-753Empty Response when using cursors after ~50 pagesNULL3
326USERGRID-754External Set of REST Tests that confirms Usergrid functionality in an execution environmentWe need a script which executes API calls to test an installation given a set of test cases to confirm that it 'works' with a bare minimum of functionality. - CRUD on entity - Query entities - CRUD on connections Add queries from here: http://apigee.com/docs/app-services/content/working-queries as test cases. 1
327USERGRID-755MVCC serialization error on reindex 2015-06-16 22:06:31,223 [http-bio-8080-exec-15] ERROR org.apache.usergrid.persistence.collection.serialization.impl.MvccLogEntrySerializationStrategyImpl- DATA CORRUPTION DETECTED when de-serializing entity with Id SimpleId{uuid=2034290e-109c-11e5-a3f5-06b0ee5388b2, type='user'}. This means the write was truncated. org.apache.usergrid.persistence.collection.exception.DataCorruptionException: Unable to read entity data org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl$EntitySerializer.fromByteBuffer(MvccEntitySerializationStrategyV3Impl.java:457) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl$EntitySerializer.fromByteBuffer(MvccEntitySerializationStrategyV3Impl.java:365) com.netflix.astyanax.serializers.AbstractSerializer.fromBytes(AbstractSerializer.java:42) com.netflix.astyanax.thrift.model.ThriftColumnImpl.getValue(ThriftColumnImpl.java:58) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl$MvccColumnParser.parseColumn(MvccEntitySerializationStrategyV3Impl.java:346) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl.lambda$load$13(MvccEntitySerializationStrategyV3Impl.java:210) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl$$Lambda$23/1238550786.call(Unknown Source) rx.Observable$5.call(Observable.java:3534) rx.internal.operators.OperatorScan$2.onNext(OperatorScan.java:104) rx.observers.SerializedObserver.onNext(SerializedObserver.java:159) rx.observers.SerializedSubscriber.onNext(SerializedSubscriber.java:95) rx.internal.operators.NotificationLite.accept(NotificationLite.java:150) rx.internal.operators.OperatorMergeMaxConcurrent$SourceSubscriber.drain(OperatorMergeMaxConcurrent.java:265) rx.internal.operators.OperatorMergeMaxConcurrent$SourceSubscriber$MergeItemSubscriber.onNext(OperatorMergeMaxConcurrent.java:325) rx.internal.operators.OperatorSubscribeOn$1$1$1.onNext(OperatorSubscribeOn.java:76) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:43) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:32) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62) rx.schedulers.ImmediateScheduler$InnerImmediateScheduler.schedule(ImmediateScheduler.java:58) rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:57) rx.internal.operators.OperatorSubscribeOn$1.onNext(OperatorSubscribeOn.java:43) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:43) rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:32) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMergeMaxConcurrent$SourceSubscriber.subscribeNext(OperatorMergeMaxConcurrent.java:147) rx.internal.operators.OperatorMergeMaxConcurrent$SourceSubscriber.onNext(OperatorMergeMaxConcurrent.java:126) rx.internal.operators.OperatorMergeMaxConcurrent$SourceSubscriber.onNext(OperatorMergeMaxConcurrent.java:68) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorBufferWithSize$1.onNext(OperatorBufferWithSize.java:103) rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:96) rx.internal.operators.OperatorBufferWithSize$1$1.request(OperatorBufferWithSize.java:88) rx.Subscriber.setProducer(Subscriber.java:177) rx.Subscriber.setProducer(Subscriber.java:171) rx.internal.operators.OperatorBufferWithSize$1.setProducer(OperatorBufferWithSize.java:74) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:47) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:33) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.subscribe(Observable.java:7585) rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:442) rx.observables.BlockingObservable.last(BlockingObservable.java:229) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl.load(MvccEntitySerializationStrategyV3Impl.java:215) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyProxyImpl.load(MvccEntitySerializationStrategyProxyImpl.java:95) org.apache.usergrid.persistence.collection.impl.EntityCollectionManagerImpl$1.call(EntityCollectionManagerImpl.java:246) org.apache.usergrid.persistence.collection.impl.EntityCollectionManagerImpl$1.call(EntityCollectionManagerImpl.java:240) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.subscribe(Observable.java:7585) rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:442) rx.observables.BlockingObservable.lastOrDefault(BlockingObservable.java:262) org.apache.usergrid.corepersistence.CpEntityManager.load(CpEntityManager.java:316) org.apache.usergrid.corepersistence.CpEntityManager.get(CpEntityManager.java:454) org.apache.usergrid.corepersistence.CpEntityManager.get(CpEntityManager.java:2344) org.apache.usergrid.management.cassandra.ManagementServiceImpl.getUserEntityByIdentifier(ManagementServiceImpl.java:1060) org.apache.usergrid.management.cassandra.ManagementServiceImpl.findUserEntity(ManagementServiceImpl.java:1132) org.apache.usergrid.management.cassandra.ManagementServiceImpl.verifyAdminUserPasswordCredentials(ManagementServiceImpl.java:1277) org.apache.usergrid.rest.management.ManagementResource.getAccessTokenInternal(ManagementResource.java:256) org.apache.usergrid.rest.management.ManagementResource.getAccessTokenPostJson(ManagementResource.java:396) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:483) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) Caused by: java.io.CharConversionException: Invalid UTF-32 character 0x7d000000(above 10ffff) at char #1, byte #7) com.fasterxml.jackson.core.io.UTF32Reader.reportInvalid(UTF32Reader.java:155) com.fasterxml.jackson.core.io.UTF32Reader.read(UTF32Reader.java:109) com.fasterxml.jackson.core.json.ReaderBasedJsonParser.loadMore(ReaderBasedJsonParser.java:131) com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:1688) com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:562) com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3031) com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2978) com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2167) org.apache.usergrid.persistence.collection.serialization.impl.MvccEntitySerializationStrategyV3Impl$EntitySerializer.fromByteBuffer(MvccEntitySerializationStrategyV3Impl.java:446) ... 145 more3
328USERGRID-756[FIXED?] NPE in CpEntityManager.validateNeed to be investigated before 2.1 release. 2015-06-16 22:47:20,361 [http-bio-8080-exec-7] ERROR org.apache.usergrid.corepersistence.CpEntityManager- Unable to load entity saparticles-fixed:e466f39a-cb97-11e4-b30c-f7cc05dca022 java.lang.NullPointerException org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:956) org.apache.usergrid.corepersistence.CpEntityManager.validate(CpEntityManager.java:942) org.apache.usergrid.corepersistence.CpEntityManager.updateProperties(CpEntityManager.java:1030) org.apache.usergrid.services.AbstractService.updateEntity(AbstractService.java:456) org.apache.usergrid.services.AbstractService.updateEntity(AbstractService.java:444) org.apache.usergrid.services.AbstractCollectionService.putItemByName(AbstractCollectionService.java:305) org.apache.usergrid.services.AbstractService.invokeItemWithName(AbstractService.java:677) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:628) org.apache.usergrid.services.AbstractService.invoke(AbstractService.java:544) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:226) org.apache.usergrid.services.ServiceRequest.execute(ServiceRequest.java:193) org.apache.usergrid.rest.applications.ServiceResource.executeServiceRequest(ServiceResource.java:251) org.apache.usergrid.rest.applications.ServiceResource.executePutWithMap(ServiceResource.java:371) org.apache.usergrid.rest.applications.ServiceResource.executePut(ServiceResource.java:420) sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:497) com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:909) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:857) com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:811) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745)3
329USERGRID-757Exception on system/database/setupthrows duplicate unique key exception on managementserviceimpl2
330USERGRID-759SNS Topic/Queue Implementation should use the Topics.subscribe helper methidThere is a helper method which should be used to subscribe SQS queues to SNS topics.2
331USERGRID-760Merge master to two-dot-o branches & rewrite tests to use new frameworkThere's one new test from master that needs to be rewritten using the new framework: ApplicationResourceIT.applicationCollectionWithAppToken()2
332USERGRID-763Portal does not show progress indicator for registrationWhen registering/signing up the form does not indicate any change in state when the 'Register' button is pressed.3
333USERGRID-764Instance Validation ScriptWe need a script which does testing of a Usergrid instance to confirm a set of functionality works.2
334USERGRID-765Validate JMX Meters / Timers are the ones we want to be in place for measuring/monitoring performanceValidate JMS Meters / Timers are the ones we want to be in place for measuring/monitoring performance3
335USERGRID-766Deploy 2.1 Branch into an End-to-End (e2e) testing environmentDeploy 2.1 Branch into an End-to-End (e2e) testing environment, including: 1) Cassandra 2) ElasticSearch 3) Tomcat 4) Portal5
336USERGRID-767WebApplicationException / Uncaught Exception embarassmentThe following exception is returned for many cases, including a parse error and if an endpoint doesn't support the attempted HTTP method. We need more descriptive messages in these cases. {"error":"web_application","timestamp":1435156493000,"duration":0,"exception":"javax.ws.rs.WebApplicationException"}3
337USERGRID-768Bad characters in query causes 500 response; expecting 400?If I make the following call: https://api.usergrid.com/org/app/collection?ql=where%20thirdPropertyTypeInt%20%3E%2030000%20&&%20thirdPropertyTypeInt%20%3C%2040000&limit=10 I get the response: {code} { "error": "null_pointer", "timestamp": 1435176848310, "duration": 1, "exception": "java.lang.NullPointerException" } {code} The cause is that the ampersands in the query string aren't url encoded (should be %26 instead of &). I would expect that if the query string is bad, we should be returning a 400 Bad Request, rather than an ominous NullPointerException/500.3
338USERGRID-769ACL not working as expected?When setting following ACL rule /assets/* user has access to /assets/xxx but also to /assets meaning that he can list all the assets. I expected that when allowing /assets/* it would only allow /assets/xxx3
339USERGRID-771Add a separate email address for approving new Org Admin users which can be different from approving new OrgsCurrently the email address appears to be the same for approving new orgs and new admins. We need a separate configurable email address for each. If this setting is blank, the admin email can be used instead. This needs to be fixed in 2.0 as well as 2.1. 2.0 first please :)3
340USERGRID-772Characterize the performance of 2.1We need to characterize the performance of 2.1. This will involve using: - Varying numbers of tomcats -- we can start with 20 and progressively add more into a load balancer to see the performance impact - Varying ES cluster size - starting with 6 and going up to 20 -- we can start with 20 nodes and only have the indexes targeted to a subset - Varying ES bucket/shard counts - starting with 10 - Varying Cassandra sizes - starting with 6 and going up to 12 General approach should be: 1) Start with 2 tomcats, 6 ES and 6 C* 2) Add +2 Tomcats iteratively until the performance starts to not increase (implying that either ES or C* is the limiting factor) 3) Add +3 ES nodes and test again. Document impact. 4) Add +3 C* nodes, remove -3 ES nodes and test again. Document impact. 5) Add +3 C* nodes and +3 ES nodes and test again. Document impact. 6) Add +2 Tomcats and see the slope of the TPS graph 7) Add +3 C* - (now at 12) and test again 8) Test iteratively with +2 Tomcats until performance plateaus, up to 20 Tomcats8
341USERGRID-773Column Querying IssuesFrom http://community.apigee.com/questions/2432/error-analysis-of-response-when-trying-to-curl-que.html#answer-2454. I have seen similar issues. I am using the org devnexus and the app 2015, which should be publicly visible. In the UI, if I set Path to /sessions and Query to select name,title,room where title contains 'Java', I get no response and no error. However, if I capture the query made by the browser (https://api.usergrid.com/devnexus/2015/sessions?ql=select%20name%2Ctitle%2Croom%20where%20title%20contains%20%27Java%27%20order%20by%20created%20desc&access_token=YWMtuzeubNpKEeSm49lERp5XKQAAAUylS2HCr3MkIzCtKZHIYLT1VUiiygN9uUs) using Chrome dev tools, the same query works with curl. You can see the same result with something as simple as select title.3
342USERGRID-774Exception being masked in ClientCredentialsSecurityFilterAn exception is being masked and causing issues in a deployment. Here is the class and line. We should be more specific on what exceptions we catch and at least log them. https://github.com/apache/incubator-usergrid/blob/49ae4ac5b8d5d77e90e6e6c6e9d8b299a5423863/stack/rest/src/main/java/org/apache/usergrid/rest/security/shiro/filters/ClientCredentialsSecurityFilter.java#L65 3
343USERGRID-775API returns 500 for missing Http MethodsNULL3
344USERGRID-776Queries without 'ql' return a Null Pointer instead of a meaningful messagehttp://api.usergrid.com/vta/sandbox/stops?select * where location within 3 of 37.415449732, -121.920367767 - this will return an 500/NPE when it should be a 4xx, and preferably have a better error message.3
345USERGRID-777Refactor App Info Migration to make it functionalNULL3
346USERGRID-778Limit on Queries results in a highly variable number of resultsThis behavior is observed on a build from the two-dot-o branch with commit ID 2d1c8b8ac7b20b63a11d83adca56839d8b409cca. For example, limit=2 gives you 1 sometimes, limit=750 gives anywhere from 625 to 749. For example this script: {code} #!/bin/bash for count in `seq 1 10` do curl -s "https://example.com/appservices/testorg/sandbox/scmocks?limit=750" > file${count} grep uuid file${count} | wc rm file${count} done {code} Produces these results: 685 2055 36305 750 2250 39750 749 2247 39697 742 2226 39326 750 2250 39750 749 2247 39697 747 2241 39591 744 2232 39432 750 2250 39750 749 2247 39697 A different count every time. 3
347USERGRID-779Convert SNS client to use the SNSAsyncClientNULL1
348USERGRID-780Implement buffer for re-index eventsCreate configurable re-index buffer property and implement in ReIndexService.1
349USERGRID-785Unable to run 2.0 to 2.1 data migration on large data set.Attempted to run entity data migration in (UG 2.0 to UG 2.1 ) with a instance having 1.1 million entities in a single collection, within a single application. This causes the following exception on cassandra: {code} ERROR [ReadStage:16] 2015-07-02 23:33:29,720 SliceQueryFilter.java (line 206) Scanned over 100000 tombstones in ug_migrate_test.Graph_Source_Node_Edges; query aborted (see tombstone_failure_threshold) ERROR [ReadStage:27] 2015-07-02 23:33:39,723 SliceQueryFilter.java (line 206) Scanned over 100000 tombstones in ug_migrate_test.Graph_Source_Node_Edges; query aborted (see tombstone_failure_threshold) ERROR [ReadStage:27] 2015-07-02 23:33:39,723 CassandraDaemon.java (line 258) Exception in thread Thread[ReadStage:27,5,main] java.lang.RuntimeException: org.apache.cassandra.db.filter.TombstoneOverwhelmingException org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2016) {code} which results in the following exception in usergrid: {code} 2015-07-02 23:33:59,680 [Index migrate data formats] ERROR org.apache.usergrid.rest.MigrateResource- Unable to migrate data java.lang.RuntimeException: Unable to connect to casandra org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.advance(MultiRowColumnIterator.java:190) org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.hasNext(MultiRowColumnIterator.java:122) org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardsColumnIterator.hasNext(ShardsColumnIterator.java:65) org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupColumnIterator.advance(ShardGroupColumnIterator.java:120) org.apache.usergrid.persistence.graph.serialization.impl.shard.impl.ShardGroupColumnIterator.hasNext(ShardGroupColumnIterator.java:68) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:66) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:38) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:84) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:71) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:38) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:96) rx.Subscriber.setProducer(Subscriber.java:177) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:47) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:33) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorMerge$InnerSubscriber.emit(OperatorMerge.java:676) rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:586) rx.internal.operators.OperatorMerge$InnerSubscriber.emit(OperatorMerge.java:676) rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:586) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorFilter$1.onNext(OperatorFilter.java:54) rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:84) rx.internal.operators.OperatorMerge$MergeSubscriber.handleScalarSynchronousObservableWithRequestLimits(OperatorMerge.java:280) rx.internal.operators.OperatorMerge$MergeSubscriber.handleScalarSynchronousObservable(OperatorMerge.java:243) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:176) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:84) org.apache.usergrid.persistence.collection.impl.EntityCollectionManagerImpl$1.call(EntityCollectionManagerImpl.java:248) org.apache.usergrid.persistence.collection.impl.EntityCollectionManagerImpl$1.call(EntityCollectionManagerImpl.java:240) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:84) rx.internal.operators.OperatorMerge$InnerSubscriber.emit(OperatorMerge.java:676) rx.internal.operators.OperatorMerge$InnerSubscriber.onNext(OperatorMerge.java:586) rx.internal.operators.OperatorFilter$1.onNext(OperatorFilter.java:54) rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:96) rx.Subscriber.setProducer(Subscriber.java:177) rx.Subscriber.setProducer(Subscriber.java:171) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:47) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:33) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) rx.internal.operators.OperatorBufferWithSize$1.onCompleted(OperatorBufferWithSize.java:119) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:75) org.apache.usergrid.persistence.core.rx.ObservableIterator.call(ObservableIterator.java:38) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorMerge$MergeSubscriber.handleNewSource(OperatorMerge.java:215) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:185) rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:120) rx.internal.operators.OnSubscribeFromIterable$IterableProducer.request(OnSubscribeFromIterable.java:96) rx.Subscriber.setProducer(Subscriber.java:177) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:47) rx.internal.operators.OnSubscribeFromIterable.call(OnSubscribeFromIterable.java:33) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable$1.call(Observable.java:144) rx.Observable$1.call(Observable.java:136) rx.Observable.unsafeSubscribe(Observable.java:7495) rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62) rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Caused by: com.netflix.astyanax.connectionpool.exceptions.OperationTimeoutException: OperationTimeoutException: [host=10.16.4.135(10.16.4.135):9160, latency=5001(30012), attempts=6]TimedOutException() com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:171) com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65) com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28) com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151) com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:119) com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:338) com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:532) org.apache.usergrid.persistence.core.astyanax.MultiRowColumnIterator.advance(MultiRowColumnIterator.java:187) ... 146 more Caused by: TimedOutException() org.apache.cassandra.thrift.Cassandra$multiget_slice_result.read(Cassandra.java:10480) org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) org.apache.cassandra.thrift.Cassandra$Client.recv_multiget_slice(Cassandra.java:673) org.apache.cassandra.thrift.Cassandra$Client.multiget_slice(Cassandra.java:657) com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:538) com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:535) com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60) ... 152 more {code}3
350USERGRID-787'order by created desc' causing no results in location query (affects 1.0, not sure about 2.0)Making the following API call returns no results: {code}https://api.usergrid.com/org/app/collection?ql=location%20within%201000%20of%2051.73213%2C%20-1.20631%20order%20by%20created%20desc{code} but this one works: {code}https://api.usergrid.com/org/app/collection?ql=location%20within%201000%20of%2051.73213%2C%20-1.20631{code} Definitely affects 1.0, not tested against 2.x3
351USERGRID-788New ExportApp toolProvide new tool that exports an app: 1. Writes two types of files: entities and connections 2. Each line of output files will be one complete JSON object 3. Use RxJava for multithreading of reads and of writes1
352USERGRID-790Can't delete entities in collection using singularized noun (affects 1.0, untested on 2.X)1. Create a collection, e.g. "food" 2. The collection will be automatically pluralized to "foods" 3. Create an entity in the "foods" collection 4. Attempt to delete "/food/[uuid]" and the following error is thrown: {code} { "error": "service_resource_not_found", "timestamp": ..., "duration": 0, "exception": "org.apache.usergrid.services.exceptions.ServiceResourceNotFoundException", "error_description": "Service resource not found" } {code}3
353USERGRID-792If the 'name' of an entity contains a colon it cannot be retrieved by nameFor example, https://usergrid.com/{org}/{app}/{collection}/jeff:west yields: { "error": "null_pointer", "timestamp": 1436487047250, "duration": 0, "exception": "java.lang.NullPointerException" }3
354USERGRID-797Org (and presumably App) credentials can't create connections by defaultIf you try to create a new connection using org or app credentials, it will fail with: {code} { "duration": 1, "error": "unauthorized", "error_description": "Subject does not have permission [applications:post:uuid:/col1/uuid/relationship/uid]", "exception": "org.apache.shiro.authz.UnauthorizedException", "timestamp": "...." } {code} The only workaround seems to be to give {code}POST /**{code} to the Guest role. Expected behavior: app and org credentials should (out of the box) both be able to create and delete connections, as well as retrieve all depths of them: {code}GET /collection/uuid/verb GET /collection/uuid/connecting/verb GET /collection/uuid/connections/verb{code} etc. In short, all tests in the URAP integration_tests should pass on a net new instance without having to set Guest role permissions to {code}/**{code}3
355USERGRID-800Test and Confirm cross-region message delivery for 2.0 in an execution environmentNULL2
356USERGRID-816Implement the ability to track the storage used by an appDepends on counters getting fixed3
357USERGRID-831Confirm 2.1 LICENSE file has requisite versions and module namesNULL3
358USERGRID-832[SPIKE] Investigate and propose 2-3 approaches for indexing specific attributes of documentsPlease investigate and propose a few different options for explicit indexing. Things to consider: 1) How other databases handle this 2) Ability to index all occurrences of a given field name - for example, index 'cat' whether it is a root property, pets.cat, owners.cat, etc. 5
359USERGRID-833Implement script for configuring Grafana dashboards based on new JMX countersNULL3
360USERGRID-840Complete Multi Region ES management implementationWhen indexes are created on setup, as well as aliases assigned on application creation, this is only performed in the local region. We need to create indexes in all regions during an index administration operation, as well as create aliases. We will also probably want to make our aliases and underlying indexes identical, in order to allow for ease of administration regardless of region. We need to account for two cases: 1) Secondary region does not already have ES indexes configured - need to create the indexes and aliases 2) Secondary region already has ES installed and configured (indexes existing) - only need to create aliases 1
361USERGRID-851ImportAdmins tool should merge organizations of duplicate usersSome older Usergrid systems may have multiple accounts with the same username or email address, something which is not allowed and problematic now. When such users are imported into a new system by the ImportAdmins program we can "repair" such users by merging the duplicates and ensure that the resulting single user account has the union of organization in the one or more duplicate users.3
362USERGRID-854[SPIKE] Look at 'Fix X test' and close if neededNULL2
363USERGRID-858Notifications POST throwing NPE{code}curl -X POST -d '{ "payloads": { "apple-dev": "Hello World!" } }' 'https://api.usergrid.com/brandon.apigee/baas-integration-tests/devices/*/notifications?client_id=redacted&client_secret=redacted'{code} throws: {code}{ "error": "runtime", "timestamp": 1436981233645, "duration": 0, "exception": "java.lang.RuntimeException", "error_description": "java.lang.NullPointerException" }{code} 1) According to docs, it should work: http://apigee.com/docs/app-services/content/creating-and-managing-notifications 2) If it doesn't and isn't supposed to, it shouldn't throw an NPE. Note that a notifier with a sandbox cert has been created with the name 'apple-dev'. --- UPDATE It does work with {code}/devices;ql=select */notifications{code} we should address this but also update the documentation to reflect the way in which it works. --- UPDATE It also appears that although the NPE is returned via the API, a notification *is* created in /notifications. In this particular test, there were no devices in the /devices collection: {code}{ "uuid": "11c5e62a-2a84-11e5-af28-65b8f0f35f5f", "type": "notifications", "created": 1436918242690, "modified": 1436918242690, "payloads": { "apple-dev": "Hello World!" }, "debug": false, "state": "CREATED", "metadata": { "path": "/notifications/11c5e62a-2a84-11e5-af28-65b8f0f35f5f", "collections": { "receipts": "/notifications/11c5e62a-2a84-11e5-af28-65b8f0f35f5f/receipts" } } }{code}2
364USERGRID-860QL for contains 'text*' does not appear to workselect * where title contains 'ta*' does not work (at all) as documented here: http://apigee.com/docs/app-services/content/working-queries3
365USERGRID-861Test and Profile performance of ExportApp toolThis tool is designed to export from 1.0 with the intent of using the resulting entities and collections in 2.x and higher. However, this data will be imported using the API so it is not import-to-2.x specific.5
366USERGRID-862Confirm that date fields are not created in 2.1 NULL1
367USERGRID-863Confirm that using where tokens in queries works in 2.1 Doesn't work in 2.0 but can't reproduce in 2.1. Try to reproduce. 1
368USERGRID-869Asset creation when using S3BinaryStore doesn't functionWhen creating an asset the following call works in 1.0 usergrid. curl -X POST -F name='example' -F file=@example.jpg 'https://localhost:8080/temp/test?access_token=<token>' properly uploading the asset. In 2.1 however the following gets returned. {"error":"uncaught","timestamp":1437415007666,"duration":0,"error_description":"Internal Server Error","exception":"org.apache.usergrid.rest.exceptions.UncaughtException","error_id":"b0c5930c-2f08-11e5-818e-060beb1b9051"}% 5
369USERGRID-872Fix and prune Usergrid tools in 2.1The Usergrid tools are broken in a variety of ways in the two-dot-o and two-dot-o-dev brands. The tools do not compile. Many rely on outdated 1.0 concepts. Many are obsolete. 1. Delete any obsolete tools 2. Make all tools compile3
370USERGRID-875Test and Validate Multi-Region Indexing1) Ensure that the indexes (including buckets) can get created in region n+1 2) Ensure that read/write aliases get created in region n+1 3) Ensure that queries in both regions return the same data 4) Attempt to quantify the latency of a PUT in region A making it to Region B and being indexed in region B Coming out of this we need a guide for setting up region N+11
371USERGRID-876[SPIKE] Review 'ignored tests'Hopefully there's nothing bad in there.3
372USERGRID-877Start Usergrid graduation process1. update status page to include all users 2. create DRAFT Top Level Project resolution 3. start discussion on dev list3
373USERGRID-878Investigate Serialization issue for NotificationsNULL3
374USERGRID-880Refactor Serialization in Core and move it down to corepersistencemove all serialization down to low levels5
375USERGRID-881Remove Fields and Map to Entity Conversions from Corewe convert from a map to entity and back 2 - 3 times for every read and write. We should remove these type conversions8
376USERGRID-882Release 2.0 with Admin email fixesNULL3
377USERGRID-889Convert Usergrid docs & Update for 2.0: Introduction & Getting StartedNULL3
378USERGRID-890Convert Usergrid docs & Update for 2.0: Data Storage, Queries and ConnectionsNULL3
379USERGRID-891Convert Usergrid docs & Update for 2.0: Push NotificationsNULL3
380USERGRID-892Convert Usergrid docs & Update for 2.0: Security & Auth and User MgmtNULL3
381USERGRID-893Convert Usergrid docs & Update for 2.0: Asset & File Mgmt and GeolocationNULL3
382USERGRID-894Convert Usergrid docs & Update for 2.0: Counters & EventsNULL3
383USERGRID-895Convert Usergrid docs & Update for 2.0: Org & App Management, User ManagementNULL3
384USERGRID-896Convert Usergrid docs & Update for 2.0: REST end-points, Models and ReferenceNULL5
385USERGRID-897Convert Usergrid docs & Update for 2.0: Usergrid SDKsNULL5
386USERGRID-898Add proper error checking to Asset Code along with error checking tests.NULL3
387USERGRID-899Validate migrate_entity_data script produces consistent migrations.Validate that https://github.com/apache/incubator-usergrid/blob/two-dot-o-dev/stack/scripts/migrate_entity_data.py produces consistent entity data migrations. 1) put data into UG build from latest in two-dot-o 2) stand up UG build from two-dot-o-dev pointing to same keyspace (different queue names) and run /system/database/setup 3) from UG running two-dot-o-dev, run migrate_entity_data.py 4) read data from UG two-dot- 4) once finished, read entities from UG two-dot-o and UG two-dot-o-dev and ensure there is an exact match 1
388USERGRID-900Re-index using AmazonAsyncEventService should support updated datesee implemented example for InMemoryAsyncEventService org/apache/usergrid/corepersistence/asyncevents/EventBuilderImpl.java:166 This filters out based on modified/updated date for the events to be indexed. Need to ensure AmazonAsyncEventService consumers implement something similar OR look to have the producer filter out the events and not even drop on the AWS queues.3
389USERGRID-901Fix CorePersistant Properties and Stack Properties to not overlap when using Aws SDKThey both use the AWS sdk but have the possibility to use different versions. This possibility should be eliminated in the future by consolidating which sdk is pulled.2
390USERGRID-902Unable to do case sensitive queries using '='Lets say you have two values {"name":"cow"} and {"name":"Cow"} When you do a query on name = "cow". We get both "cow" and "Cow" back in the results. Equals should be a case sensitive search. ergo we should only be returning "cow"3
391USERGRID-903Cannot query and return entities that end in a space.not being able to query and return entities which end in a space "value " If I submit a value with {"name":"value "} then do a search on name="value" I do not retrieve the value. This should be checked to see if we do get the value back if we query on name="value ". If we do then we have a couple of options. We could trim the string so people cannot have values with space on the end, or could mark as a non issue since you really want a search of contains or something similar. "=" should be saved for equalities. 3
392USERGRID-904Encodings in values are not being returned from query engine.Values like “%7BaccountId%7D�? are being put into usergrid but they can no longer be queried out.3
393USERGRID-907Batch interval in seconds not honoredThe following property is present in the configuration of counter batch submits. {code} usergrid.counter.batch.interval=30 {code} This is not honored. This needs fixed in the 2.0 and 2.1 branch.3
394USERGRID-908run specific migrationNULL1
395USERGRID-909Evaluate removing cache from edge shard managementCurrently, using caches with the proposal phase for new shards has caused large numbers of tombstones. Review the algorithm, and determine the cause of the additional shard proposals. Usergrid currently performs 5 shard lookups on write and most miss the cache. We'll change the following *Shard allocation* Shard allocation will not perform a propose + check phase. The following algorithm will occur. This must occur at EACH_QUORUM to ensure that we have consensus across regions. Any node will propose a new shard pivot. This pivot will have compacted set to false, and will have a column timestamp of a new timeuuid in micros The proposing node will then read back all compacted = false shard pivots. The shard pivot with the minimum timestamp will be retained, all others will be deleted as proposals. This will function similarly to distributed read locks in Cassandra. Lowest proposed value always wins. Allocation is complete *Shard compaction* After a successful proposal allocation, the allocating node will copy all edges from the previously compacted shard into the new shard. It will continue to copy until no new edges are returned on read. Once read has been completed, all edges that were copied can be deleted from the previous shard. This target shard will be marked as compacted *Consistency* When performing consistency, we will need a higher consistency level. To ensure that shards exist worldwide, proposal + selection should occur at EACH_QUORUM. Standard reads can occur as LOCAL_QUORUM 5
396USERGRID-912Upgrade Shiro to be current versionIn order to support NIO. Current version is ancient. 1
397USERGRID-914Blog post for 10k TPS and horizontal scalabilityNULL2
398USERGRID-918Add qparam to skip retrieving metadata and graph edgesIf the qparam is not there, use current/default behavior. If the qparam is there then skip pulling the extra data which will result in better performance.8
399USERGRID-921Remove duplicate add to collection on write pathTodd's spaghetti comments on write path which results in 2x writes to cassandra ops.3
400USERGRID-926Include distances in geoquery responsesThe best place for this would be in the metadata of the results, in metadata.distance perhaps.3
401USERGRID-929Test the creation of entities which have an OldUUID to migrate from 1.0 to 2.1NULL1
402USERGRID-933[SPIKE] Understand what remains and break up tasks for migrating properties from files on Tomcat to Org-level in the management appNULL3
403USERGRID-936Unit Test for MapToEntityConverter.processListForField to test for 2d array and infinite loop 2015-08-25 00:01:10,246 [http-bio-8080-exec-313] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- org.apache.usergrid.rest.exceptions.UncaughtException Server Error (500) org.apache.usergrid.rest.exceptions.UncaughtException: java.lang.StackOverflowError at org.apache.usergrid.rest.exceptions.AbstractExceptionMapper.toResponse(AbstractExceptionMapper.java:59) at org.apache.usergrid.rest.exceptions.ThrowableMapper.toResponse(ThrowableMapper.java:37) at com.sun.jersey.spi.container.ContainerResponse.mapException(ContainerResponse.java:480) at com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException(ContainerResponse.java:417) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1477) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:927) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:875) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:829) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.StackOverflowError at org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148) at org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148) at org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148) at org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148) at org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148)1
404USERGRID-937Nested Arrays in documents result in infinite loop / stack overflowNeed to investigate and decide on an approach. In addition, do we support arrays with elements of varying types? org.apache.usergrid.persistence.model.entity.MapToEntityConverter.processListForField(MapToEntityConverter.java:148)1
405USERGRID-938Create a tool with the capability to wipe an application from CassandraGiven an App ID, we need to be able to wipe the data from Cassandra. This can be standalone.3
406USERGRID-939Style generated Docs like the WebsiteCurrently our Sphinx-generated docs adopt the look and feel of the ReadTheDocs website, which is not bad but it does not look like the rest of our website. We should style the docs so that the use the same header and footer as our existing website and use the same colors, fonts, etc.3
407USERGRID-940Generated docs should include Client SDK docsThe generated docs should include our existing Client SDK docs, which already exist in Markdown format within the Usergrid Git repo.3
408USERGRID-941Spike: Investigate using Java Annotations to generate SwaggerNULL5
409USERGRID-944Prototype using Spark to perform functions on data in CassandraAs we discussed we'd like to see about using Spark for things such as: 1) migrating orgs from one keyspace to another 2) deleting all records for an app from Cassandra3
410USERGRID-945Perform load testing of new shard management implWe'll want to load test this with our existing entity writes during heavy load. We need to ensure shard allocation can keep up, and that we have a consistent view of the shards. We need to test the following scenarios # Write over 10 million entities in a single collection, single region only # Write over 10 million entities concurrently in 2 regions # Write over 10 million entities concurrently in 2 regions, and fail 1 of the regions. The system should still function, just not allocate new shards. 5
411USERGRID-950Remove Cassandra Keyspace name from Index Naming StrategyNULL1
412USERGRID-989Create a multi region lock ability for shard allocationWith the new shard proposal algorithm, we need the ability to either 1) Perform routing aware shard allocation across multi regions. This ensures that we 5
413USERGRID-991Fix tests after Jersey 2.0 upgradeNULL5
414USERGRID-992Enable Org-level propertiesNULL5
415USERGRID-993Separate database system and system bootstrapA call to /system/database/setup currently performs a database setup, ES setup, and then proceeds to migrate data. This should be split into 2 separate events. They should be the following. /system/database/setup creates all column families and ES indexes /system/database/bootstrap runs a datamigration manager, then creates all bootstrap data required for usergrid to function3
416USERGRID-1000get apm/apigeeMobileConfig throws exception when its not set2015-09-04 22:34:01,420 [http-bio-8080-exec-22] ERROR org.apache.usergrid.rest.exceptions.AbstractExceptionMapper- Uncaught Exception java.lang.IllegalArgumentException: JSON source MUST not be null at com.sun.jersey.api.json.JSONWithPadding.<init>(JSONWithPadding.java:90) at org.apache.usergrid.rest.applications.ApplicationResource.getAPMConfig(ApplicationResource.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:927) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:875) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:829) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.usergrid.rest.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:92) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)3
417USERGRID-1005Add Background processing for delete applicationNULL2
418USERGRID-1007Implement a cache for Shiro & permissionsHow will a cache help improve performance? * Reduce load on Tomcat, ES and C* caused by permissons calculation. Do we need a distributed cache? * YES: Todd says we tried non-distributed cache (w/EhCache) and the caching was not effective (bad hit/mss ratio). Implementation ideas: * Use Cassandra column family implementation * Use Hazelcast cache * Use EhCache configured for distributed operation How much should we expect performance to improve? How can we test this? * JUnit + metrics collection in our cache implementation Simplest approach seems to be Cassandra. Here's a design: h1. Add new Core Persistence Cache module This is a new module that depends only on the Core Persistence Common module (and maybe Model?) and provides a generic cache interface that is "segmented" by a cache scope. When you put or get things you specify the cache scope, and it is possible to invalidate all things within a cache scope with one call. {code} public interface SegmentedCache<K,V> { /** Put value into segment with ttl */ void put( CacheScope scope, K key, V value, long ttl ); /** Get value from segment */ V get( CacheScope segment, K key ); /** Delete all cache data of all types and of all scopes */ void invalidate(); /** Delete all cache data within specified segment */ void invalidate( CacheScope segment ); } {code} Caches are stored in Cassandra *Usergrid_Cache* column family: * Row-key is applicationId * Column key is toString() of K key * Value is serialized V value object h1. Plugin our Cache into Shiro at the REST module level To plugin to Shiro, we provide implementations of the Shiro *CacheManager* and *Cache* interfaces that are backed by the Cache module. When permissions are updated, we need need to ensure that the old cached permissions are invalidated. So we will modify the *EntityManager* revoke and grant methods to call invalidate for the application's cache scope. 5
419USERGRID-1013Automate upload of api baas 2.1 build to s3NULL3
420USERGRID-1015Test Notification Migration NULL3
421USERGRID-1016Remove UUID field encoding in our ES fieldsWe shouldn't encode a UUID as a specific UUID field type, since we're encoding it as a string. To keep our index tuples simple, and support both queries, we should store it as a string type. We should support the following formats. {code} select * where myUuid = 'aaa65d8a-16a6-11e5-86ea-7b4f7ff44431' {code} As well as {code} select * where myUuid = aaa65d8a-16a6-11e5-86ea-7b4f7ff44431 {code} Notice the addition of single quotes in the first query, vs the parsing as a UUID type in the second query. This will enable users to use both versions, but does not affect performance. 3
422USERGRID-1017Create a job to clean unused edgesWe need a system job that audits edges and ensures they point to an entity. If not, the edge should be removed.3
423USERGRID-1018Create a job to remove duplicate connections and fix connection write logicA bug in 2.0 has led to multiple edges on a connection. We need a job that scans all connections, and keeps the lowest timestamp on the connection. Other edges should be deleted. 3
424USERGRID-1022remove batching in index processingNULL1
425USERGRID-1023Delete from collection does not use SQS or the event serviceNULL3
426USERGRID-1024Fix EntityMapSerialization in EntityToMapConverter to remove type serializationEntityToMapConverter line 143 injects binary objects into our serialized maps. Probably a good idea to change this to just map structures instead. we want to remove the type info. see attached3
427USERGRID-1026Logic to detect (N+1)-d arrays catches a case where there is no 2-d arrayThis entity is not fully indexed with the error message below. It does not have a (direct) 2-d array. It does have nested arrays, however. 2015-09-18 03:20:28,190 [Usergrid-RxIOPool-151] WARN org.apache.usergrid.persistence.index.impl.EntityMappingParser- Encountered 2 collections consecutively. N+1 dimensional arrays are unsupported, only arrays of depth 1 are supported { "type" : "example", "name" : "8980031", "created" : 1442527975706, "modified" : 1442547269098, "FacilityName" : "ALICES RESTAURANT", "LastUpdateOn" : "9/17/2015", "Locations" : [ { "Addresses" : [ { "Zip" : "94062", "AddressLine1" : "17288 Skyline Blvd", "LatNbr" : "37.386916", "LongNbr" : "-122.265383", "State" : "CA", "AddrUseCd" : "PH", "City" : "WOODSIDE" } ], "Phones" : [ { "PhnUseCd" : "PH", "PhoneNbr" : "123-456-3534" } ], "LocSeq" : "1", "PracLocStrtDt" : "1/1/9999", "PracLocEndDt" : "3/31/9999", "Accepting" : false, "PracLocSpecialties" : [ { "SpcltyPrimInd" : "Y", "SpcltyDsc" : "PANCAKES", "SpcltyCd" : "BRKF" } ] } ], "TheId" : 8980031, "ProviderType" : "FAC", "PvdrEndDt" : "3/31/2008", "PvdrSpecialties" : [ { "SpcltyPrimInd" : "Y", "SpcltyDsc" : "PANCAKES", "SpcltyCd" : "BRKF" } ] }1
428USERGRID-1027Occational MIsses on Indexing into ElasticsearchOccasionally applications still aren't being indexed into elasticsearch. Need to investigate why these might be occuring. 8
429USERGRID-1029When PUTing by uuid or name, add the entry into entity_id_sets column familyWhen a PUT occurs in 1.0, then entry to Entity_Id_Sets is not re-inserted. We need to ensure it is re-written. 3
430USERGRID-1031Test and confirm Usergrid 1.0 works with Cassandra 2.11) Test locally 2) Confirm in e2e 2
431USERGRID-1032Test and confirm Usergrid 2.1 works with Cassandra 2.1NULL2
432USERGRID-1033Test and confirm Usergrid 2.0 works with Cassandra 2.1NULL1
433USERGRID-1035Fix NoAwsCreds rule to work outside of core. The no aws creds rule call doesn't work outside of core. It fails in service and rest when using the one based in core. 2
434USERGRID-1037Refactor asset aws sdk classRefactor the class so that it doesn't need to retrieve the s3 client everytime. also we need to use the usergrid credentials class so that we can match how we connect to sqs. That way the code is more consistent. 2
435USERGRID-10382.1-release: fix index batching from SQS readerNULL3
436USERGRID-1046remove index and deindex requests that do not go through sqs e.g. CandidateIdFiltermove all these requests to sqs. any piece of code that calls indexproducer.put() has to be changed to call the index service and call to ES asynchronously through sqs 3
437USERGRID-1048When reading messages from SQS an elevated consistency may be needed with latent replicationOccasionally Cassandra multi-region replication lags during high load. During this high load, messages fail to process because it cannot read the entity from Cassandra. Rather than fail, we should retry with a high consistency level of quorum in all regions to ensure we replicate the data correctly. 3
438USERGRID-1051When ElasticSearch rejects a Search request Usergrid should not return 200 OK with an empty arrayCurrently if the search queue is full a 200 response is returned with an empty entity array.1
439USERGRID-1052After large deletions, graph seek time becomes slowAfter a large number of deletions, 100k +, seek time on graph becomes very slow when searching collections. Identify the issue, and implement a fix based off of sha a09485a3a5ac8b4217b294f4754ea8a70a7ec447 5
440USERGRID-1053[SPIKE] Audit TODOs in the codeWe need to audit the TODO statements in the code. For each we must: 1) Check if there is an existing ticket - If there is no ticket create one 2) Update the TODO statement with the link to the ticket 3) Prioritize based on: - Will this generate a P1 case with a customer? - What is the usability impact of the product? Once we have audited these we will prioritize and schedule fixes5
441USERGRID-1058Ability to generate a master Jacoco coverage reportIn addition to separate Jacoco reports for each module, we should also be able to generate a master Jacoco report.3
442USERGRID-1067Finish 2.1 release testingWe need to test the following issues to verify they've been fixed. Multi region writes and index syncing Multi region mass collection deletes: Delete the entire collection, then ensure that seek times are fast with no query language. 3
443USERGRID-1068Create a per app migration scriptCreate a python script that will allow users to upgrade from 2.0 to 2.1 per application. This will ultimately allow an incremental deployment across an existing 2.0 cluster, without user impact. Template for full system script that already exists: https://github.com/apache/usergrid/blob/2.1-release/stack/scripts/migrate_entity_data.py It should allow for the following migration process: tenant 1 1) setup new 2.1 tomcat 2) run migration bootstrap 3) migrate appinfo 4) reindex mgmt app only 5) run de-dup for whatever APP 6) reindex whatever APP 7) add 2.1 tomcats to customer ELB after 1-6 completed for all customer apps 8) run delta re-index/migration for all customer apps tenant 2 1) reset appinfo version 2) migrate app info 3) reindex mgmt app 4) run de-dup for whatever APP 5) reindex whatever app 7) add 2.1 tomcats to customer ELB after 1-6 completed for all customer apps 8) run delta re-index/migration for all customer apps after all apps migration 1) migrate entity data 2
444USERGRID-1069implement build metrics in jenkinsNULL1
445USERGRID-1071Test iOS push notificationsNULL1
446USERGRID-1072Test Android push notificationsNULL1
447USERGRID-1073Test Windows phoneNULL1
448USERGRID-1074[SPIKE] Evaluate read repair of unique values in 1.0NULL1
449USERGRID-1075Create Repair Tool to address Service Resource Exceptions from users who cannot be queried for their dataFind a way to resolve the state where a user may still have an entity ref but is not able to be queried or retrieved without uuid.5
450USERGRID-1076Create a Repair tool that will address null pointer exceptions on entity returns and then properly handle them500 null pointer exceptions return when data is present in an entity reference but not returned from cassandra while this may be a data consistency issue an investigation needs to be done. That said in the mean time a tool should be created that either can retrieve the data and restore it ( update ) or to delete the left over entity reference. 5
451USERGRID-1077Create a repair task in usergrid to handle instances of multiple INDEXES when they are detectedThings that need to be addressed when they are found. EntityManagerImpl.java line 1135 if ( properties == null ) { logger.error( "Error deserializing entity with key {} entity probaby doesn't exist, where did this key come from?", key ); continue; } UUID id = uuid( properties.get( PROPERTY_UUID ) ); String type = string( properties.get( PROPERTY_TYPE ) ); if ( ( id == null ) || ( type == null ) ) { logger.error( "Error retrieving entity with key {}, no type or id deseriazable, where did this key come from?", key ); continue; } Then in EntityManagerImpl getUUIDsForUniqueProperty //shouldn't happen, but it's an error case if ( cols.size() > 1 ) { logger.error( "INDEX CORRUPTION: More than 1 unique value exists for entities in ownerId {} of type {} on " + "property {} with value {}", new Object[] { ownerEntityId, collectionNameInternal, propertyName, propertyValue } ); } ^ Need to address both issues on read repair path such that we no longer need to worry about users seeing them. For the second issue we should be able to just call delete UniqueProperty with the entity unique value in question. Just make this happen in the backend. 8
452USERGRID-1079Add 2 endpoints for password hash migrationIt is currently possible to migrate from a 1.0 installation to a 2.1 installation via a RESTful client. However, due to the inability to securely move password hashes, application user's passwords are not retained. Add the following. # In the 1.x branch, add the ability to retrieve the password hash. This should only be allowed by the superuser. # In 2.1-release, add the ability to write the password hash to an application user. This should only be allowed by the superuser. Note that the reason this is only allowed as a superuser is that we want to disable this functionality by default. Any UG installation that is public facing should not have superuser enable. This allows us to disable this functionality in environments that are publicly available environments. 5
453USERGRID-1085Create RPM Distributable for Usergrid 2.1NULL5
454USERGRID-1086Errors/Failed index events during mgmt app re-indexThese errors/failed index events do not seem to cause any operating issues with Usergrid, but cause messages to fall into dead letter queue and increase in ERROR log statements. 2015-10-30 22:31:48,552 [Usergrid-RxIOPool-152] ERROR org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService- Received empty index sequence message:(6c53a75c-6351-4f2b-8e86-e0176c0bebcd), body:({"edgeIndexEvent":{"creationTime":1446244308487,"sourceRegion":"us-east-1","applicationScope":{"application":{"uuid":"b6768a08-b5d5-11e3-a495-11ddb1de66c8","type":"application"}},"entityId":{"uuid":"b6768a08-b5d5-11e3-a495-11ddb1de66c8","type":"org_application"},"edge":{"sourceNode":{"uuid":"b6768a08-b5d5-11e3-a495-11ddb1de66c8","type":"application"},"type":"zzzcollzzz|org_applications","targetNode":{"uuid":"b6768a08-b5d5-11e3-a495-11ddb1de66c8","type":"org_application"},"timestamp":136655371084730107}}}) 2015-10-30 22:31:48,553 [Usergrid-RxIOPool-152] ERROR org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService- Failed to index message: 6c53a75c-6351-4f2b-8e86-e0176c0bebcd java.lang.Exception: Received empty index sequence. at org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService.lambda$callEventHandlers$21(AmazonAsyncEventService.java:357) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService.callEventHandlers(AmazonAsyncEventService.java:372) at org.apache.usergrid.corepersistence.asyncevents.AmazonAsyncEventService.lambda$null$23(AmazonAsyncEventService.java:695) at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55) at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:46) at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:35) at rx.Observable$2.call(Observable.java:162) at rx.Observable$2.call(Observable.java:154) at rx.Observable.unsafeSubscribe(Observable.java:7710) at rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:62) at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) at rx.schedulers.ExecutorScheduler$ExecutorSchedulerWorker.run(ExecutorScheduler.java:98) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)3
455USERGRID-1091AbstractReadGraphFilter produces and queues empty queue operationsWhen read repair is triggered, it is possible that there are now deIndexRequests to be queued. Currently we're queueing empty operations and need to filter this properly.1
456USERGRID-1092Test and design for ElasticSearch running on Tomcat nodesThis can become a coordination task with other teams as well as executing it ourselves - for example with Kris Bhandare5
457USERGRID-1093Load test in-memory queue with current consumer and queue implementationNULL3
458USERGRID-1099Order of operations error in Query Execution I have a test that shows that queries of this form work: C = c and ( A > a or B = false ) But the very same query formed like this will not work properly: ( A > a or B = false ) and C = c 5
459USERGRID-1100POST /users/{user}/devices/notifications does not work.NULL1
460USERGRID-1101Unique Value Cleanup in 2.1 - cleaning up old columns for old versionsEnsure code makes it in to prevent wide rows in unique_values CF due to old versions not getting cleaned up 3
461USERGRID-1106Fix test: clientCredentialsFlowWithHeaderAuthorization()Currently this test is ignored in Usergrid master branch: ApplicationResourceIT.clientCredentialsFlowWithHeaderAuthorization() If it is still relevant we should fix it and un-ignore it.3
462USERGRID-1113Improve NotificationsService test coverageCurrently many of the NotificationsServiceIT and NotifiersServiceIT test are either broken or incomplete because we were unable to create the mocks we need to test the code. We should revisit this and figure out how to get better test coverage for notifications in general. Search the code for USERGRID-1113 to find the tests that should be unignored, fixed, replace or deleted.2
463USERGRID-1115Fix AdminUsersIT testsSix of the AdminUsersIT tests are ignored because they interfere with other tests and other reasons. Figure out how to run those tests serially or some other way to test admin user features.5
464USERGRID-1118Fix EventsResource and ITThe EventsResource was broken during two-dot-o development and the tests in EventsResourceIT were marked ignore. Fix the EventsResource and re-enable the tests.3
465USERGRID-1119Fix GroupResourceIT.postGroupActivity() testNULL3
466USERGRID-1124Internal "size" within entity metadata is not displayed when doing a GET by entity name This request does not come back with size in metadata object. GET /org/app/collection/entityName {code} { "action" : "get", "application" : "ec2df3b3-6e0e-11e5-9798-0223e9015bf9", "params" : { }, "path" : "/regions", "uri" : "https://testing", "entities" : [ { "uuid" : "2dcf254c-8ef5-11e5-87c2-069a7df27e87", "type" : "region", "name" : "region6", "created" : 1447961939135, "modified" : 1447961939135, "metadata" : { "path" : "/regions/2dcf254c-8ef5-11e5-87c2-069a7df27e87" } } ], "timestamp" : 1447976782774, "duration" : 77, "organization" : "org", "applicationName" : "app" } {code} However, get with the entity's UUID does return the metadata size. GET /org/app/collection/2dcf254c-8ef5-11e5-87c2-069a7df27e87 {code} { "action" : "get", "application" : "ec2df3b3-6e0e-11e5-9798-0223e9015bf9", "params" : { }, "path" : "/regions", "uri" : "https://testing", "entities" : [ { "uuid" : "2dcf254c-8ef5-11e5-87c2-069a7df27e87", "type" : "region", "name" : "region6", "created" : 1447961939135, "modified" : 1447961939135, "metadata" : { "path" : "/regions/2dcf254c-8ef5-11e5-87c2-069a7df27e87", "size" : 334 } } ], "timestamp" : 1447977010952, "duration" : 36, "organization" : "org", "applicationName" : "app" } {code}1
467USERGRID-1182Support GCM 3.0 for Android Push NotificationsNULL2
468USERGRID-1183Support APNS VoIP Push NotificationsIncludes supporting an additional certificate type VoIP Services Certificate. Also we should enhance cert upload validation, etc. when creating notifiers in Usergrid.2
469USERGRID-1186Add DELETE for Admin Users- Will need superuser credentials. - Will delete admin user. - Will remove all connections from said admin user. ( might automatically be done. needs verifying. ) -Verify can be recreated 2
470USERGRID-1187Ensure that PUT or POST does not create duplicate org association connections. Doing PUT creates a duplicate edge on /management/orgs/org/users/email for existing users who were already on the org, the POST command to /management/orgs/org/users/email creates a duplicate edge 2
471USERGRID-1191Create 2.1 Release for ApacheWe need to create a 2.1 release for Apache. Would be nice to have a binary WAR distribution.3
472USERGRID-1203Allow certs in notifiers to be updated and store their details in the entity.NULL1
473USERGRID-1209GET on missing Asset file in S3 generates 500 error not 404NULL1
474USERGRID-1210Fix AwsAssetResourceITFix all the broken asset tests3
475USERGRID-1211Migrate locks from Hector to AstyanaxImplement locks and a lock manager using Astyanax driver.3
476USERGRID-1238Error occuring when clicking link to activate organizationGetting an email when trying to activate an organization: {"error":"unauthorized","timestamp":1405399644281,"duration":0,"exception":"org.apache.usergrid.rest.exceptions.SecurityException","error_description":"No organization access authorized"} Steps to reproduce - sign up for an account, then click activation link in email that goes to sys admin When the link is clicked then the following message pops up. Ideally it should be a rest response that denotes show the activation was successful.3
477USERGRID-1239Add smtp message-id to logsAdd message-id to the smtp logs. http://stackoverflow.com/questions/5555119/sending-email-using-smtp-and-setting-the-message-id This will help us track whether usergrid is failing to send registration email or if it is the mail service. There also needs to be additional logging enabled to track what stages the emails goes through if it is needed.3
478USERGRID-1244Upgrade pushy library to use latest version (HTTP/2 APNS implementation) Additional system requirements will need to be met (https://github.com/relayrides/pushy#system-requirements): 1) require openssl to be installed on servers running Usergrid and have specific compiled versions of usergrid for the supported platforms for running Usergrid (os version, etc.). http://netty.io/wiki/forked-tomcat-native.html 2) require an additional jar file added to the boot classpath of the jvm (alpn-boot) which is specific to the JDK version being used It may make sense to see about having the ability to disable push so that no push notification services are bootstrapped in the event someone wants to use Usergrid but does not require push and does not want to manage these additional system requirements. Option 1 preferred at the moment as having OpenSSL installed is an easy requirement and and the platform configurations for http://netty.io/wiki/forked-tomcat-native.html seem limited.2
479USERGRID-1256Introduce Datastax Java Driver in effort to migration away from AstyanaxNULL1
480USERGRID-1267Simplify Central SSO: eliminate ext token validation end-pointInstead of requiring that clients first validate tokens obtained from an external "Central SSO" system, when Usergrid is configured for using Central SSO, it should attempt to verify any unknown token itself. This can be implemented by moving the validate external token logic from the ManagementResource and into the TokenService.3
481USERGRID-1268Proof of Concept: use Akka for Unique Value enforcementBy using an distributed Actor System, we can ensure that only one thread accesses any one unique value at a time. I have a working proof of concept of this that writes to Cassandra (via Java Native Driver). Initial design is here: https://docs.google.com/document/d/1o9okFgNb_c1RY0eIcZMhijhxE9v7BxHYDiuI8BqkLgU/edit# Proof of Concept (POC) code is here: https://github.com/snoopdave/akka_poc1 Presentation that explains the POC is here: https://docs.google.com/presentation/d/11ARSPZ6IAxOYSTo4jcmEdnzXvOgVGNGslznmVtLOZjY/edit?usp=sharing 8
482USERGRID-1272Set a property to turn off usergrid's dependency on elasticsearchHave a property that can tell Usergrid whether we want to use elasticsearch or not. If we don't we shouldn't have any errors, should just work as is with only graph searches . 5
483USERGRID-1275Allow usergrid.api.url.base to be set when creating organizations.Allowing usergrid.api.url.base this field to be set or set differently so it is not forgotten when you are running usergrid. 2