Building

Started building at 2021/12/24 02:00:32
Calculating base
Updating mirror
Basing run on 7.1.0-1977 671ed59f2
Updating tree for run 24.12.2021-02.00
gocb is at 635fa6c, changes since last good build: none
libcouchbase is at f6822fc, changes since last good build: none
indexing is at 8d3522b, changes since last good build: none
kv_engine is at cda4d73, changes since last good build: none
ns_server is at 8590527, changes since last good build: none
couchstore is at 0bf3a4b, changes since last good build: none
couchdb is at fca2e6b, changes since last good build: none
testrunner is at a23e488, changes since last good build: none
Switching eventing to unstable
eventing is at ee5930d, changes since last good build: 
 ee5930db MB-50179: Use vbucket forward map for master node calculation during vbucket movement
 dff6ad00 MB-48538, MB-49435, MB-47111: Http layer refactor
 a8cc6056 MB-48538: Add new audit ids for different events
Switching eventing-ee to unstable
eventing-ee is at 8fcd80e, changes since last good build: none
Pulling in uncommitted change 160397 at refs/changes/97/160397/3
Total 23 (delta 15), reused 17 (delta 15)
[unstable 2405e8c7] MB-100 : Do not fail n1ql timeout test if we are catching timeout exceptions successfully
 Author: abhijpes 
 Date: Mon Aug 30 08:24:38 2021 +0530
 1 file changed, 2 insertions(+), 2 deletions(-)
Pulling in uncommitted change 163449 at refs/changes/49/163449/1
Total 48 (delta 27), reused 39 (delta 27)
[unstable f7456fdf] MB-100: collect data when test fails
 Author: AnkitPrabhu 
 Date: Tue Oct 12 14:02:31 2021 +0530
 1 file changed, 2 insertions(+)
Pulling in uncommitted change 166698 at refs/changes/98/166698/3
Total 4 (delta 3), reused 4 (delta 3)
[unstable b9102279] MB-48334 : Fail rebalance unless all functions deployed on current encryption level
 Author: abhijpes 
 Date: Wed Dec 1 14:51:29 2021 +0530
 1 file changed, 9 insertions(+)
Building cmakefiles and deps
Building main product

Testing

Started testing at 2021/12/24 02:14:52
Testing mode: rebalance,n1ql,functional
Starting server
Starting server: attempt 1

Functional tests

2021/12/24 02:16:20 Started functional
Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:39320->127.0.0.1:9000: read: connection reset by peer
Quota setup Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:39320->127.0.0.1:9000: read: connection reset by peer
2021/12/24 02:16:29 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/24 02:16:35 Error in adding nodes : , response: {"otpNode":"n_1@127.0.0.1"}
2021/12/24 02:16:35 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 02:16:40 Removing node(s): [] from the cluster
2021/12/24 02:16:45 Rebalance progress: 0
2021/12/24 02:16:50 Rebalance progress: 0.1265914351851852
2021/12/24 02:16:55 Rebalance progress: 4.64771412037037
2021/12/24 02:17:00 Rebalance progress: 9.25925925925926
2021/12/24 02:17:05 Rebalance progress: 12.73148148148148
2021/12/24 02:17:10 Rebalance progress: 17.43344907407407
2021/12/24 02:17:15 Rebalance progress: 20.83333333333333
2021/12/24 02:17:20 Rebalance progress: 25.4810474537037
2021/12/24 02:17:25 Rebalance progress: 100
2021/12/24 02:17:28 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:17:43 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:17:58 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:18:13 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:18:28 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:18:43 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:18:58 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:19:13 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:19:28 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:19:43 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:19:58 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:20:13 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:20:28 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:20:43 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:20:58 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:21:13 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:21:28 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:21:43 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
2021/12/24 02:21:58 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:0}]
=== RUN   TestAdvancedGetOps
2021/12/24 02:22:16 Deploying app: TestAdvancedGetOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:22:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps stored in temp store"
}
2021/12/24 02:22:22 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/24 02:22:27 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/24 02:22:32 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/24 02:22:37 Waiting for app: TestAdvancedGetOps to get deployed
2021/12/24 02:22:37 App: TestAdvancedGetOps got deployed
2021/12/24 02:22:37 Starting bucket ops to source bucket
2021/12/24 02:22:42 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 02:22:42 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 02:22:42 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:22:42 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:22:47 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/24 02:22:52 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/24 02:22:57 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/24 02:23:02 Waiting for app: TestAdvancedGetOps to get un-deployed
2021/12/24 02:23:02 App: TestAdvancedGetOps got un-deployed
2021/12/24 02:23:02 No eventing-con process running
2021/12/24 02:23:02 Function: TestAdvancedGetOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedGetOps deleting in background"
}
2021/12/24 02:23:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:23:14 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:23:19 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:23:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:23:21 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:23:31 Waiting for dst bucket item count to get to: 0 curr count: 92
2021/12/24 02:23:36 Waiting for dst bucket item count to get to: 0 curr count: 92
2021/12/24 02:23:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedGetOps (80.20s)
=== RUN   TestAdvancedInsertOps
2021/12/24 02:23:36 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:23:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/24 02:23:42 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:23:47 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:23:52 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:23:57 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:23:57 App: TestAdvancedInsertOps got deployed
2021/12/24 02:23:57 Starting bucket ops to source bucket
2021/12/24 02:24:03 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/24 02:24:08 Waiting for dst bucket item count to get to: 1024 curr count: 580
2021/12/24 02:24:13 Waiting for dst bucket item count to get to: 1024 curr count: 580
2021/12/24 02:24:13 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/24 02:24:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:24:13 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:24:18 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:24:23 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:24:28 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:24:33 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:24:33 App: TestAdvancedInsertOps got un-deployed
2021/12/24 02:24:33 No eventing-con process running
2021/12/24 02:24:33 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/24 02:24:35 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:24:45 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:24:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:24:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:24:57 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:24:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:24:57 Testing insert operation on source bucket
2021/12/24 02:24:57 Deploying app: TestAdvancedInsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:24:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps stored in temp store"
}
2021/12/24 02:25:03 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:25:08 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:25:13 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:25:18 Waiting for app: TestAdvancedInsertOps to get deployed
2021/12/24 02:25:18 App: TestAdvancedInsertOps got deployed
2021/12/24 02:25:18 Starting bucket ops to source bucket
2021/12/24 02:25:24 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/24 02:25:29 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/24 02:25:34 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/24 02:25:34 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/24 02:25:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:25:34 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:25:39 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:25:44 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:25:49 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:25:54 Waiting for app: TestAdvancedInsertOps to get un-deployed
2021/12/24 02:25:54 App: TestAdvancedInsertOps got un-deployed
2021/12/24 02:25:54 No eventing-con process running
2021/12/24 02:25:54 Function: TestAdvancedInsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertOps deleting in background"
}
2021/12/24 02:25:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:26:06 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/24 02:26:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:26:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:26:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertOps (156.50s)
=== RUN   TestAdvancedUpsertOps
2021/12/24 02:26:12 Starting bucket ops to source bucket
2021/12/24 02:26:13 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:26:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/24 02:26:19 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:26:24 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:26:29 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:26:34 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:26:34 App: TestAdvancedUpsertOps got deployed
2021/12/24 02:26:34 Starting bucket ops to source bucket
2021/12/24 02:26:40 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:26:45 Waiting for dst bucket item count to get to: 0 curr count: 512
2021/12/24 02:26:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:26:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:26:45 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:26:50 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:26:55 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:27:00 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:27:05 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:27:05 App: TestAdvancedUpsertOps got un-deployed
2021/12/24 02:27:05 No eventing-con process running
2021/12/24 02:27:05 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/24 02:27:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:27:17 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:27:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:27:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:27:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:27:24 Testing upsert operation on source bucket
2021/12/24 02:27:24 Deploying app: TestAdvancedUpsertOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:27:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps stored in temp store"
}
2021/12/24 02:27:30 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:27:35 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:27:40 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:27:45 Waiting for app: TestAdvancedUpsertOps to get deployed
2021/12/24 02:27:45 App: TestAdvancedUpsertOps got deployed
2021/12/24 02:27:45 Starting bucket ops to source bucket
2021/12/24 02:27:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:27:46 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:27:46 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:27:51 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:27:56 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:28:01 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:28:06 Waiting for app: TestAdvancedUpsertOps to get un-deployed
2021/12/24 02:28:06 App: TestAdvancedUpsertOps got un-deployed
2021/12/24 02:28:06 No eventing-con process running
2021/12/24 02:28:06 Function: TestAdvancedUpsertOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedUpsertOps deleting in background"
}
2021/12/24 02:28:08 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:28:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:28:15 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:28:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedUpsertOps (128.16s)
=== RUN   TestAdvancedReplaceOps
2021/12/24 02:28:20 Starting bucket ops to source bucket
2021/12/24 02:28:21 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:28:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/24 02:28:27 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:28:32 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:28:37 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:28:42 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:28:42 App: TestAdvancedReplaceOps got deployed
2021/12/24 02:28:42 Starting bucket ops to source bucket
2021/12/24 02:28:49 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:28:54 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:28:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:28:54 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:28:54 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:28:59 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:29:04 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:29:09 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:29:14 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:29:14 App: TestAdvancedReplaceOps got un-deployed
2021/12/24 02:29:14 No eventing-con process running
2021/12/24 02:29:14 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/24 02:29:15 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:29:26 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 02:29:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:29:27 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:29:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:29:32 Testing upsert operation on source bucket
2021/12/24 02:29:32 Deploying app: TestAdvancedReplaceOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:29:33 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps stored in temp store"
}
2021/12/24 02:29:38 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:29:43 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:29:48 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:29:54 Waiting for app: TestAdvancedReplaceOps to get deployed
2021/12/24 02:29:54 App: TestAdvancedReplaceOps got deployed
2021/12/24 02:29:54 Starting bucket ops to source bucket
2021/12/24 02:29:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:29:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:29:55 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:30:00 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:30:05 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:30:10 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:30:15 Waiting for app: TestAdvancedReplaceOps to get un-deployed
2021/12/24 02:30:15 App: TestAdvancedReplaceOps got un-deployed
2021/12/24 02:30:15 No eventing-con process running
2021/12/24 02:30:15 Function: TestAdvancedReplaceOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedReplaceOps deleting in background"
}
2021/12/24 02:30:16 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:30:26 Waiting for dst bucket item count to get to: 0 curr count: 337
2021/12/24 02:30:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:30:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:30:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedReplaceOps (132.67s)
=== RUN   TestAdvancedDeleteOps
2021/12/24 02:30:33 Starting bucket ops to source bucket
2021/12/24 02:30:33 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:30:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/24 02:30:39 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:30:44 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:30:49 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:30:54 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:30:54 App: TestAdvancedDeleteOps got deployed
2021/12/24 02:30:54 Starting bucket ops to source bucket
2021/12/24 02:31:00 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:31:05 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:31:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:31:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:31:05 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:31:10 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:31:15 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:31:20 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:31:25 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:31:25 App: TestAdvancedDeleteOps got un-deployed
2021/12/24 02:31:25 No eventing-con process running
2021/12/24 02:31:25 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/24 02:31:26 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:31:36 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:31:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:31:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:31:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:31:43 Testing delete operation on source bucket
2021/12/24 02:31:43 Deploying app: TestAdvancedDeleteOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:31:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps stored in temp store"
}
2021/12/24 02:31:49 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:31:54 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:31:59 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:32:04 Waiting for app: TestAdvancedDeleteOps to get deployed
2021/12/24 02:32:04 App: TestAdvancedDeleteOps got deployed
2021/12/24 02:32:04 Starting bucket ops to source bucket
2021/12/24 02:32:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:32:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:32:05 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:32:10 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:32:15 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:32:20 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:32:25 Waiting for app: TestAdvancedDeleteOps to get un-deployed
2021/12/24 02:32:25 App: TestAdvancedDeleteOps got un-deployed
2021/12/24 02:32:25 No eventing-con process running
2021/12/24 02:32:25 Function: TestAdvancedDeleteOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedDeleteOps deleting in background"
}
2021/12/24 02:32:26 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:32:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:32:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:32:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedDeleteOps (125.62s)
=== RUN   TestEnoentAdvancedGet
2021/12/24 02:32:39 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:32:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/24 02:32:45 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:32:50 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:32:55 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:33:00 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:33:00 App: TestEnoentAdvancedGet got deployed
2021/12/24 02:33:00 Starting bucket ops to source bucket
2021/12/24 02:33:05 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 02:33:10 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 02:33:15 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 02:33:15 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 02:33:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:33:15 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:33:20 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:33:25 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:33:30 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:33:35 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:33:35 App: TestEnoentAdvancedGet got un-deployed
2021/12/24 02:33:35 No eventing-con process running
2021/12/24 02:33:35 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/24 02:33:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:33:47 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:33:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:33:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:33:59 Waiting for dst bucket item count to get to: 0 curr count: 98
2021/12/24 02:34:04 Waiting for dst bucket item count to get to: 0 curr count: 48
2021/12/24 02:34:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:34:04 Testing get enoent operation on source bucket
2021/12/24 02:34:04 Deploying app: TestEnoentAdvancedGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:34:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet stored in temp store"
}
2021/12/24 02:34:10 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:34:15 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:34:20 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:34:25 Waiting for app: TestEnoentAdvancedGet to get deployed
2021/12/24 02:34:25 App: TestEnoentAdvancedGet got deployed
2021/12/24 02:34:25 Starting bucket ops to source bucket
2021/12/24 02:34:30 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/24 02:34:30 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/24 02:34:30 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:34:30 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:34:35 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:34:40 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:34:45 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:34:50 Waiting for app: TestEnoentAdvancedGet to get un-deployed
2021/12/24 02:34:50 App: TestEnoentAdvancedGet got un-deployed
2021/12/24 02:34:50 No eventing-con process running
2021/12/24 02:34:50 Function: TestEnoentAdvancedGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedGet deleting in background"
}
2021/12/24 02:34:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:35:02 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/24 02:35:07 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/24 02:35:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:35:09 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:35:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedGet (155.01s)
=== RUN   TestEnoentAdvancedReplace
2021/12/24 02:35:14 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:35:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/24 02:35:20 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:35:25 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:35:30 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:35:35 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:35:35 App: TestEnoentAdvancedReplace got deployed
2021/12/24 02:35:35 Starting bucket ops to source bucket
2021/12/24 02:35:40 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 02:35:45 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 02:35:45 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 02:35:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:35:45 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:35:50 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:35:55 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:36:00 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:36:05 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:36:05 App: TestEnoentAdvancedReplace got un-deployed
2021/12/24 02:36:05 No eventing-con process running
2021/12/24 02:36:05 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/24 02:36:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:36:17 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:36:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:36:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:36:29 Waiting for dst bucket item count to get to: 0 curr count: 97
2021/12/24 02:36:34 Waiting for dst bucket item count to get to: 0 curr count: 47
2021/12/24 02:36:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:36:34 Testing get enoent operation on source bucket
2021/12/24 02:36:34 Deploying app: TestEnoentAdvancedReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:36:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace stored in temp store"
}
2021/12/24 02:36:40 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:36:45 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:36:50 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:36:55 Waiting for app: TestEnoentAdvancedReplace to get deployed
2021/12/24 02:36:55 App: TestEnoentAdvancedReplace got deployed
2021/12/24 02:36:55 Starting bucket ops to source bucket
2021/12/24 02:37:00 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/24 02:37:05 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/24 02:37:05 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/24 02:37:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:37:05 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:37:10 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:37:15 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:37:20 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:37:25 Waiting for app: TestEnoentAdvancedReplace to get un-deployed
2021/12/24 02:37:25 App: TestEnoentAdvancedReplace got un-deployed
2021/12/24 02:37:25 No eventing-con process running
2021/12/24 02:37:25 Function: TestEnoentAdvancedReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedReplace deleting in background"
}
2021/12/24 02:37:27 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:37:37 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/24 02:37:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:37:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:37:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedReplace (149.99s)
=== RUN   TestEnoentAdvancedDelete
2021/12/24 02:37:44 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:37:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/24 02:37:50 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:37:55 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:38:00 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:38:05 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:38:05 App: TestEnoentAdvancedDelete got deployed
2021/12/24 02:38:05 Starting bucket ops to source bucket
2021/12/24 02:38:10 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 02:38:15 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 02:38:15 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 02:38:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:38:15 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:38:20 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:38:25 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:38:30 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:38:35 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:38:35 App: TestEnoentAdvancedDelete got un-deployed
2021/12/24 02:38:35 No eventing-con process running
2021/12/24 02:38:35 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/24 02:38:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:38:47 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:38:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:38:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:38:59 Waiting for dst bucket item count to get to: 0 curr count: 95
2021/12/24 02:39:04 Waiting for dst bucket item count to get to: 0 curr count: 45
2021/12/24 02:39:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:39:04 Testing get enoent operation on source bucket
2021/12/24 02:39:04 Deploying app: TestEnoentAdvancedDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:39:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete stored in temp store"
}
2021/12/24 02:39:10 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:39:15 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:39:20 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:39:25 Waiting for app: TestEnoentAdvancedDelete to get deployed
2021/12/24 02:39:25 App: TestEnoentAdvancedDelete got deployed
2021/12/24 02:39:25 Starting bucket ops to source bucket
2021/12/24 02:39:30 Waiting for dst bucket item count to get to: 200 curr count: 0
2021/12/24 02:39:35 Waiting for dst bucket item count to get to: 200 curr count: 100
2021/12/24 02:39:35 src & dst bucket item count matched up. src bucket count: 200 dst bucket count: 200
2021/12/24 02:39:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:39:35 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:39:40 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:39:45 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:39:50 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:39:55 Waiting for app: TestEnoentAdvancedDelete to get un-deployed
2021/12/24 02:39:55 App: TestEnoentAdvancedDelete got un-deployed
2021/12/24 02:39:55 No eventing-con process running
2021/12/24 02:39:55 Function: TestEnoentAdvancedDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentAdvancedDelete deleting in background"
}
2021/12/24 02:39:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:40:07 Waiting for dst bucket item count to get to: 0 curr count: 200
2021/12/24 02:40:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:40:09 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:40:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentAdvancedDelete (149.94s)
=== RUN   TestAdvancedInsertKeyExist
2021/12/24 02:40:14 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:40:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/24 02:40:20 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:40:25 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:40:30 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:40:35 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:40:35 App: TestAdvancedInsertKeyExist got deployed
2021/12/24 02:40:35 Starting bucket ops to source bucket
2021/12/24 02:40:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:40:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:40:35 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:40:40 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:40:45 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:40:50 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:40:55 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:40:55 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/24 02:40:55 No eventing-con process running
2021/12/24 02:40:55 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/24 02:40:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:41:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:41:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:41:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:41:09 Pumping document in the destination bucket
2021/12/24 02:41:09 Starting bucket ops to source bucket
2021/12/24 02:41:09 Deploying app: TestAdvancedInsertKeyExist. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:41:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist stored in temp store"
}
2021/12/24 02:41:16 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:41:21 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:41:26 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:41:31 Waiting for app: TestAdvancedInsertKeyExist to get deployed
2021/12/24 02:41:31 App: TestAdvancedInsertKeyExist got deployed
2021/12/24 02:41:31 Starting bucket ops to source bucket
2021/12/24 02:41:36 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:41:41 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:41:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:41:41 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:41:41 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:41:46 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:41:51 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:41:56 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:42:01 Waiting for app: TestAdvancedInsertKeyExist to get un-deployed
2021/12/24 02:42:01 App: TestAdvancedInsertKeyExist got un-deployed
2021/12/24 02:42:01 No eventing-con process running
2021/12/24 02:42:01 Function: TestAdvancedInsertKeyExist delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAdvancedInsertKeyExist deleting in background"
}
2021/12/24 02:42:03 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:42:13 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:42:18 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:42:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:42:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:42:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAdvancedInsertKeyExist (130.72s)
=== RUN   TestExipryGet
2021/12/24 02:42:24 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:42:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/24 02:42:31 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:42:36 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:42:41 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:42:46 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:42:46 App: TestExipryGet got deployed
2021/12/24 02:42:46 Starting bucket ops to source bucket
2021/12/24 02:43:31 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:43:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:43:31 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:43:31 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:43:36 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:43:41 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:43:46 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:43:51 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:43:51 App: TestExipryGet got un-deployed
2021/12/24 02:43:51 No eventing-con process running
2021/12/24 02:43:51 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/24 02:43:53 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:43:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:44:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:44:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:44:05 Pumping document in the destination bucket for expiry get
2021/12/24 02:44:05 Starting bucket ops to source bucket
2021/12/24 02:44:05 Deploying app: TestExipryGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:44:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet stored in temp store"
}
2021/12/24 02:44:11 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:44:16 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:44:21 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:44:26 Waiting for app: TestExipryGet to get deployed
2021/12/24 02:44:26 App: TestExipryGet got deployed
2021/12/24 02:44:26 Starting bucket ops to source bucket
2021/12/24 02:45:12 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:45:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:45:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:45:12 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:45:17 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:45:22 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:45:27 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:45:32 Waiting for app: TestExipryGet to get un-deployed
2021/12/24 02:45:32 App: TestExipryGet got un-deployed
2021/12/24 02:45:32 No eventing-con process running
2021/12/24 02:45:32 Function: TestExipryGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryGet deleting in background"
}
2021/12/24 02:45:33 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:45:43 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:45:48 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:45:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:45:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:45:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryGet (210.81s)
=== RUN   TestExipryInsert
2021/12/24 02:45:55 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:45:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/24 02:46:01 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:46:06 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:46:11 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:46:17 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:46:17 App: TestExipryInsert got deployed
2021/12/24 02:46:17 Starting bucket ops to source bucket
2021/12/24 02:47:02 Waiting for dst bucket item count to get to: 0 curr count: 75
2021/12/24 02:47:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:47:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:47:02 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:47:07 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:47:12 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:47:17 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:47:22 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:47:22 App: TestExipryInsert got un-deployed
2021/12/24 02:47:22 No eventing-con process running
2021/12/24 02:47:22 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/24 02:47:24 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:47:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:47:31 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:47:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:47:36 Testing on the destination bucket for expiry insert
2021/12/24 02:47:36 Deploying app: TestExipryInsert. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:47:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert stored in temp store"
}
2021/12/24 02:47:42 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:47:47 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:47:52 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:47:57 Waiting for app: TestExipryInsert to get deployed
2021/12/24 02:47:57 App: TestExipryInsert got deployed
2021/12/24 02:47:57 Starting bucket ops to source bucket
2021/12/24 02:48:42 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:48:47 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:48:52 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:48:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:48:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:48:52 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:48:57 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:49:02 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:49:07 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:49:12 Waiting for app: TestExipryInsert to get un-deployed
2021/12/24 02:49:12 App: TestExipryInsert got un-deployed
2021/12/24 02:49:12 No eventing-con process running
2021/12/24 02:49:12 Function: TestExipryInsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryInsert deleting in background"
}
2021/12/24 02:49:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:49:24 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:49:29 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:49:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:49:31 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:49:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryInsert (220.66s)
=== RUN   TestExipryReplace
2021/12/24 02:49:36 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:49:37 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/24 02:49:42 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:49:47 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:49:52 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:49:57 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:49:57 App: TestExipryReplace got deployed
2021/12/24 02:49:57 Starting bucket ops to source bucket
2021/12/24 02:50:42 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:50:47 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:50:52 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:50:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:50:52 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:50:52 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:50:58 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:51:03 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:51:08 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:51:13 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:51:13 App: TestExipryReplace got un-deployed
2021/12/24 02:51:13 No eventing-con process running
2021/12/24 02:51:13 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/24 02:51:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:51:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:51:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:51:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:51:27 Testing on the destination bucket for expiry insert
2021/12/24 02:51:27 Deploying app: TestExipryReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:51:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace stored in temp store"
}
2021/12/24 02:51:33 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:51:38 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:51:43 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:51:48 Waiting for app: TestExipryReplace to get deployed
2021/12/24 02:51:48 App: TestExipryReplace got deployed
2021/12/24 02:51:48 Starting bucket ops to source bucket
2021/12/24 02:52:33 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:52:38 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:52:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:52:38 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:52:38 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:52:43 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:52:48 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:52:53 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:52:58 Waiting for app: TestExipryReplace to get un-deployed
2021/12/24 02:52:58 App: TestExipryReplace got un-deployed
2021/12/24 02:52:58 No eventing-con process running
2021/12/24 02:52:58 Function: TestExipryReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryReplace deleting in background"
}
2021/12/24 02:53:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:53:10 Waiting for dst bucket item count to get to: 0 curr count: 53
2021/12/24 02:53:15 Waiting for dst bucket item count to get to: 0 curr count: 53
2021/12/24 02:53:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:53:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:53:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryReplace (225.64s)
=== RUN   TestExipryUpsert
2021/12/24 02:53:22 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:53:23 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/24 02:53:28 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:53:33 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:53:38 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:53:43 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:53:43 App: TestExipryUpsert got deployed
2021/12/24 02:53:43 Starting bucket ops to source bucket
2021/12/24 02:54:28 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:54:33 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:54:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:54:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:54:33 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:54:38 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:54:43 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:54:48 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:54:53 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:54:53 App: TestExipryUpsert got un-deployed
2021/12/24 02:54:53 No eventing-con process running
2021/12/24 02:54:53 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/24 02:54:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:55:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:55:02 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:55:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:55:07 Testing on the destination bucket for expiry insert
2021/12/24 02:55:07 Deploying app: TestExipryUpsert. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:55:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert stored in temp store"
}
2021/12/24 02:55:13 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:55:18 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:55:23 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:55:28 Waiting for app: TestExipryUpsert to get deployed
2021/12/24 02:55:28 App: TestExipryUpsert got deployed
2021/12/24 02:55:28 Starting bucket ops to source bucket
2021/12/24 02:56:14 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:56:19 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 02:56:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:56:19 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:56:19 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:56:24 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:56:29 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:56:34 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:56:39 Waiting for app: TestExipryUpsert to get un-deployed
2021/12/24 02:56:39 App: TestExipryUpsert got un-deployed
2021/12/24 02:56:39 No eventing-con process running
2021/12/24 02:56:39 Function: TestExipryUpsert delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestExipryUpsert deleting in background"
}
2021/12/24 02:56:41 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:56:51 Waiting for dst bucket item count to get to: 0 curr count: 85
2021/12/24 02:56:56 Waiting for dst bucket item count to get to: 0 curr count: 85
2021/12/24 02:56:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:56:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:57:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestExipryUpsert (220.78s)
=== RUN   TestCasReplace
2021/12/24 02:57:02 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:57:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/24 02:57:09 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:57:14 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:57:19 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:57:24 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:57:24 App: TestCasReplace got deployed
2021/12/24 02:57:24 Starting bucket ops to source bucket
2021/12/24 02:57:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:57:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:57:24 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:57:29 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:57:34 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:57:39 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:57:44 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:57:44 App: TestCasReplace got un-deployed
2021/12/24 02:57:44 No eventing-con process running
2021/12/24 02:57:44 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/24 02:57:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:57:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:57:53 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:57:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:57:58 Testing on the destination bucket for upsert with cas
2021/12/24 02:57:58 Starting bucket ops to source bucket
2021/12/24 02:57:58 Deploying app: TestCasReplace. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:57:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace stored in temp store"
}
2021/12/24 02:58:04 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:58:09 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:58:14 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:58:19 Waiting for app: TestCasReplace to get deployed
2021/12/24 02:58:19 App: TestCasReplace got deployed
2021/12/24 02:58:19 Starting bucket ops to source bucket
2021/12/24 02:58:24 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:58:29 Waiting for dst bucket item count to get to: 0 curr count: 93
2021/12/24 02:58:34 Waiting for dst bucket item count to get to: 0 curr count: 43
2021/12/24 02:58:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:58:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:58:34 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:58:40 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:58:45 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:58:50 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:58:55 Waiting for app: TestCasReplace to get un-deployed
2021/12/24 02:58:55 App: TestCasReplace got un-deployed
2021/12/24 02:58:55 No eventing-con process running
2021/12/24 02:58:55 Function: TestCasReplace delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasReplace deleting in background"
}
2021/12/24 02:58:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:59:06 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 02:59:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:59:08 Context: Bucket flush, response code: 200 dump: 
2021/12/24 02:59:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasReplace (130.70s)
=== RUN   TestCasDelete
2021/12/24 02:59:13 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 02:59:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/24 02:59:19 Waiting for app: TestCasDelete to get deployed
2021/12/24 02:59:24 Waiting for app: TestCasDelete to get deployed
2021/12/24 02:59:29 Waiting for app: TestCasDelete to get deployed
2021/12/24 02:59:34 Waiting for app: TestCasDelete to get deployed
2021/12/24 02:59:34 App: TestCasDelete got deployed
2021/12/24 02:59:34 Starting bucket ops to source bucket
2021/12/24 02:59:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 02:59:35 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 02:59:35 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 02:59:40 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 02:59:45 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 02:59:50 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 02:59:55 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 02:59:55 App: TestCasDelete got un-deployed
2021/12/24 02:59:55 No eventing-con process running
2021/12/24 02:59:55 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/24 02:59:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:00:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:00:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:00:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:00:09 Testing on the destination bucket for delete with cas
2021/12/24 03:00:09 Starting bucket ops to source bucket
2021/12/24 03:00:09 Deploying app: TestCasDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:00:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete stored in temp store"
}
2021/12/24 03:00:15 Waiting for app: TestCasDelete to get deployed
2021/12/24 03:00:20 Waiting for app: TestCasDelete to get deployed
2021/12/24 03:00:25 Waiting for app: TestCasDelete to get deployed
2021/12/24 03:00:30 Waiting for app: TestCasDelete to get deployed
2021/12/24 03:00:30 App: TestCasDelete got deployed
2021/12/24 03:00:30 Starting bucket ops to source bucket
2021/12/24 03:00:35 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:00:40 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:00:45 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:00:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:00:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:00:45 http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 03:00:50 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 03:00:55 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 03:01:00 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 03:01:05 Waiting for app: TestCasDelete to get un-deployed
2021/12/24 03:01:05 App: TestCasDelete got un-deployed
2021/12/24 03:01:05 No eventing-con process running
2021/12/24 03:01:05 Function: TestCasDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCasDelete deleting in background"
}
2021/12/24 03:01:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:01:17 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:01:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:01:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:01:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCasDelete (130.83s)
=== RUN   TestCountersIncrement
2021/12/24 03:01:24 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 03:01:30 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 03:01:30 Removing node(s): [] from the cluster
2021/12/24 03:01:35 Rebalance progress: 100
2021/12/24 03:01:35 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:01:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/24 03:01:41 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:01:46 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:01:51 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:01:56 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:01:56 App: TestCountersIncrement got deployed
2021/12/24 03:01:56 Starting bucket ops to source bucket
2021/12/24 03:02:03 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/24 03:02:08 Waiting for dst bucket item count to get to: 1024 curr count: 866
2021/12/24 03:02:13 Waiting for dst bucket item count to get to: 1024 curr count: 866
2021/12/24 03:02:13 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/24 03:02:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:02:18 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:02:23 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:02:28 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:02:33 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:02:33 App: TestCountersIncrement got un-deployed
2021/12/24 03:02:33 No eventing-con process running
2021/12/24 03:02:33 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/24 03:02:35 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:02:45 Waiting for dst bucket item count to get to: 0 curr count: 1025
2021/12/24 03:02:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:02:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:02:56 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 03:02:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:02:56 Deploying app: TestCountersIncrement. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:02:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement stored in temp store"
}
2021/12/24 03:03:03 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:03:08 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:03:13 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:03:18 Waiting for app: TestCountersIncrement to get deployed
2021/12/24 03:03:18 App: TestCountersIncrement got deployed
2021/12/24 03:03:18 Starting bucket ops to source bucket
2021/12/24 03:03:24 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/24 03:03:29 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/24 03:03:34 Waiting for dst bucket item count to get to: 2048 curr count: 1024
2021/12/24 03:03:34 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/24 03:03:34 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:03:39 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:03:44 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:03:49 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:03:54 Waiting for app: TestCountersIncrement to get un-deployed
2021/12/24 03:03:54 App: TestCountersIncrement got un-deployed
2021/12/24 03:03:54 No eventing-con process running
2021/12/24 03:03:54 Function: TestCountersIncrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersIncrement deleting in background"
}
2021/12/24 03:03:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:04:06 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/24 03:04:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:04:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:04:17 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/24 03:04:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:04:17 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/24 03:04:23 Rebalance progress: 100
--- PASS: TestCountersIncrement (178.66s)
=== RUN   TestCountersDecrement
2021/12/24 03:04:23 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 03:04:23 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 03:04:23 Removing node(s): [] from the cluster
2021/12/24 03:04:28 Rebalance progress: 100
2021/12/24 03:04:28 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:04:29 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/24 03:04:34 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:04:39 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:04:44 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:04:49 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:04:49 App: TestCountersDecrement got deployed
2021/12/24 03:04:49 Starting bucket ops to source bucket
2021/12/24 03:04:55 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/24 03:05:00 Waiting for dst bucket item count to get to: 1024 curr count: 0
2021/12/24 03:05:05 Waiting for dst bucket item count to get to: 1024 curr count: 512
2021/12/24 03:05:05 src & dst bucket item count matched up. src bucket count: 1024 dst bucket count: 1024
2021/12/24 03:05:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:05:11 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:05:16 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:05:21 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:05:26 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:05:26 App: TestCountersDecrement got un-deployed
2021/12/24 03:05:26 No eventing-con process running
2021/12/24 03:05:26 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/24 03:05:27 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:05:37 Waiting for dst bucket item count to get to: 0 curr count: 513
2021/12/24 03:05:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:05:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:05:49 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 03:05:54 Waiting for dst bucket item count to get to: 0 curr count: 512
2021/12/24 03:05:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:05:54 Deploying app: TestCountersDecrement. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:05:55 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement stored in temp store"
}
2021/12/24 03:06:00 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:06:05 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:06:10 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:06:15 Waiting for app: TestCountersDecrement to get deployed
2021/12/24 03:06:15 App: TestCountersDecrement got deployed
2021/12/24 03:06:15 Starting bucket ops to source bucket
2021/12/24 03:06:22 Waiting for dst bucket item count to get to: 2048 curr count: 0
2021/12/24 03:06:22 src & dst bucket item count matched up. src bucket count: 2048 dst bucket count: 2048
2021/12/24 03:06:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:06:27 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:06:32 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:06:37 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:06:42 Waiting for app: TestCountersDecrement to get un-deployed
2021/12/24 03:06:42 App: TestCountersDecrement got un-deployed
2021/12/24 03:06:42 No eventing-con process running
2021/12/24 03:06:42 Function: TestCountersDecrement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCountersDecrement deleting in background"
}
2021/12/24 03:06:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:06:54 Waiting for dst bucket item count to get to: 0 curr count: 2048
2021/12/24 03:06:59 Waiting for dst bucket item count to get to: 0 curr count: 1024
2021/12/24 03:06:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:07:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:07:10 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/24 03:07:15 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/24 03:07:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:07:15 Removing node(s): [http://127.0.0.1:19002 http://127.0.0.1:19003] from the cluster
2021/12/24 03:07:21 Rebalance progress: 100
--- PASS: TestCountersDecrement (178.00s)
=== RUN   TestCacheGet
2021/12/24 03:07:21 Deploying app: TestCacheGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:07:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet stored in temp store"
}
2021/12/24 03:07:27 Waiting for app: TestCacheGet to get deployed
2021/12/24 03:07:32 Waiting for app: TestCacheGet to get deployed
2021/12/24 03:07:37 Waiting for app: TestCacheGet to get deployed
2021/12/24 03:07:42 Waiting for app: TestCacheGet to get deployed
2021/12/24 03:07:42 App: TestCacheGet got deployed
2021/12/24 03:07:42 Starting bucket ops to source bucket
2021/12/24 03:07:47 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/24 03:07:52 Waiting for dst bucket item count to get to: 20 curr count: 0
2021/12/24 03:07:57 Waiting for dst bucket item count to get to: 20 curr count: 5
2021/12/24 03:08:02 Waiting for dst bucket item count to get to: 20 curr count: 5
2021/12/24 03:08:07 Waiting for dst bucket item count to get to: 20 curr count: 17
2021/12/24 03:08:12 Waiting for dst bucket item count to get to: 20 curr count: 17
2021/12/24 03:08:12 src & dst bucket item count matched up. src bucket count: 20 dst bucket count: 20
2021/12/24 03:08:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:08:17 Waiting for app: TestCacheGet to get un-deployed
2021/12/24 03:08:22 Waiting for app: TestCacheGet to get un-deployed
2021/12/24 03:08:27 Waiting for app: TestCacheGet to get un-deployed
2021/12/24 03:08:32 Waiting for app: TestCacheGet to get un-deployed
2021/12/24 03:08:32 App: TestCacheGet got un-deployed
2021/12/24 03:08:32 No eventing-con process running
2021/12/24 03:08:32 Function: TestCacheGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheGet deleting in background"
}
2021/12/24 03:08:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:08:44 Waiting for dst bucket item count to get to: 0 curr count: 20
2021/12/24 03:08:49 Waiting for dst bucket item count to get to: 0 curr count: 10
2021/12/24 03:08:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:08:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:09:01 Waiting for dst bucket item count to get to: 0 curr count: 19
2021/12/24 03:09:06 Waiting for dst bucket item count to get to: 0 curr count: 19
2021/12/24 03:09:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheGet (105.34s)
=== RUN   TestCacheRYOW
2021/12/24 03:09:06 Deploying app: TestCacheRYOW. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:09:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW stored in temp store"
}
2021/12/24 03:09:12 Waiting for app: TestCacheRYOW to get deployed
2021/12/24 03:09:17 Waiting for app: TestCacheRYOW to get deployed
2021/12/24 03:09:22 Waiting for app: TestCacheRYOW to get deployed
2021/12/24 03:09:27 Waiting for app: TestCacheRYOW to get deployed
2021/12/24 03:09:27 App: TestCacheRYOW got deployed
2021/12/24 03:09:27 Starting bucket ops to source bucket
2021/12/24 03:09:32 Waiting for dst bucket item count to get to: 50 curr count: 0
2021/12/24 03:09:37 Waiting for dst bucket item count to get to: 50 curr count: 25
2021/12/24 03:09:42 Waiting for dst bucket item count to get to: 50 curr count: 25
2021/12/24 03:09:42 src & dst bucket item count matched up. src bucket count: 50 dst bucket count: 50
2021/12/24 03:09:42 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:09:47 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/24 03:09:53 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/24 03:09:58 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/24 03:10:03 Waiting for app: TestCacheRYOW to get un-deployed
2021/12/24 03:10:03 App: TestCacheRYOW got un-deployed
2021/12/24 03:10:03 No eventing-con process running
2021/12/24 03:10:03 Function: TestCacheRYOW delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheRYOW deleting in background"
}
2021/12/24 03:10:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:10:14 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:10:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:10:16 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:10:26 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:10:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheRYOW (80.28s)
=== RUN   TestCacheExpiry
2021/12/24 03:10:26 Deploying app: TestCacheExpiry. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:10:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry stored in temp store"
}
2021/12/24 03:10:32 Waiting for app: TestCacheExpiry to get deployed
2021/12/24 03:10:37 Waiting for app: TestCacheExpiry to get deployed
2021/12/24 03:10:42 Waiting for app: TestCacheExpiry to get deployed
2021/12/24 03:10:47 Waiting for app: TestCacheExpiry to get deployed
2021/12/24 03:10:47 App: TestCacheExpiry got deployed
2021/12/24 03:10:47 Starting bucket ops to source bucket
2021/12/24 03:10:53 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:10:58 Waiting for dst bucket item count to get to: 100 curr count: 3
2021/12/24 03:11:03 Waiting for dst bucket item count to get to: 100 curr count: 3
2021/12/24 03:11:08 Waiting for dst bucket item count to get to: 100 curr count: 86
2021/12/24 03:11:13 Waiting for dst bucket item count to get to: 100 curr count: 86
2021/12/24 03:11:13 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:11:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:11:18 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/24 03:11:23 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/24 03:11:28 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/24 03:11:33 Waiting for app: TestCacheExpiry to get un-deployed
2021/12/24 03:11:33 App: TestCacheExpiry got un-deployed
2021/12/24 03:11:33 No eventing-con process running
2021/12/24 03:11:33 Function: TestCacheExpiry delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiry deleting in background"
}
2021/12/24 03:11:35 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:11:45 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:11:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:11:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:11:56 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:11:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiry (90.25s)
=== RUN   TestCacheExpiryCustom
2021/12/24 03:11:56 Deploying app: TestCacheExpiryCustom. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:11:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom stored in temp store"
}
2021/12/24 03:12:03 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/24 03:12:08 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/24 03:12:13 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/24 03:12:18 Waiting for app: TestCacheExpiryCustom to get deployed
2021/12/24 03:12:18 App: TestCacheExpiryCustom got deployed
2021/12/24 03:12:18 Starting bucket ops to source bucket
2021/12/24 03:12:23 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:12:28 Waiting for dst bucket item count to get to: 100 curr count: 3
2021/12/24 03:12:33 Waiting for dst bucket item count to get to: 100 curr count: 3
2021/12/24 03:12:38 Waiting for dst bucket item count to get to: 100 curr count: 86
2021/12/24 03:12:43 Waiting for dst bucket item count to get to: 100 curr count: 86
2021/12/24 03:12:43 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:12:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:12:48 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/24 03:12:53 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/24 03:12:58 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/24 03:13:03 Waiting for app: TestCacheExpiryCustom to get un-deployed
2021/12/24 03:13:03 App: TestCacheExpiryCustom got un-deployed
2021/12/24 03:13:03 No eventing-con process running
2021/12/24 03:13:03 Function: TestCacheExpiryCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheExpiryCustom deleting in background"
}
2021/12/24 03:13:05 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:13:15 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:13:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:13:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:13:27 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:13:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheExpiryCustom (90.32s)
=== RUN   TestCacheOverflowCustom
2021/12/24 03:13:27 Deploying app: TestCacheOverflowCustom. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:13:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom stored in temp store"
}
2021/12/24 03:13:33 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/24 03:13:38 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/24 03:13:43 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/24 03:13:48 Waiting for app: TestCacheOverflowCustom to get deployed
2021/12/24 03:13:48 App: TestCacheOverflowCustom got deployed
2021/12/24 03:13:48 Starting bucket ops to source bucket
2021/12/24 03:13:53 Waiting for dst bucket item count to get to: 35 curr count: 0
2021/12/24 03:13:58 Waiting for dst bucket item count to get to: 35 curr count: 3
2021/12/24 03:14:03 Waiting for dst bucket item count to get to: 35 curr count: 3
2021/12/24 03:14:08 Waiting for dst bucket item count to get to: 35 curr count: 24
2021/12/24 03:14:13 Waiting for dst bucket item count to get to: 35 curr count: 30
2021/12/24 03:14:18 Waiting for dst bucket item count to get to: 35 curr count: 34
2021/12/24 03:14:23 Waiting for dst bucket item count to get to: 35 curr count: 34
2021/12/24 03:14:23 src & dst bucket item count matched up. src bucket count: 35 dst bucket count: 35
2021/12/24 03:14:23 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:14:28 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/24 03:14:33 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/24 03:14:38 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/24 03:14:43 Waiting for app: TestCacheOverflowCustom to get un-deployed
2021/12/24 03:14:43 App: TestCacheOverflowCustom got un-deployed
2021/12/24 03:14:43 No eventing-con process running
2021/12/24 03:14:43 Function: TestCacheOverflowCustom delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCacheOverflowCustom deleting in background"
}
2021/12/24 03:14:45 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:14:55 Waiting for dst bucket item count to get to: 0 curr count: 5
2021/12/24 03:14:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:14:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:15:07 Waiting for dst bucket item count to get to: 0 curr count: 35
2021/12/24 03:15:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCacheOverflowCustom (100.35s)
=== RUN   TestStrictMode
2021/12/24 03:15:07 Deploying app: TestStrictMode. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:15:08 Context: Post to main store, response code: 422 dump: {
 "name": "ERR_HANDLER_COMPILATION",
 "code": 27,
 "description": {
  "area": "handlerCode",
  "column_number": 63,
  "compile_success": false,
  "description": "        destination[meta.id] = 00000000000000000000000000000101;\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nSyntaxError: Octal literals are not allowed in strict mode.\n",
  "index": 132,
  "language": "JavaScript",
  "line_number": 4
 }
}
--- PASS: TestStrictMode (1.07s)
=== RUN   TestDataTypes
2021/12/24 03:15:08 Deploying app: TestDataTypes. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:15:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes stored in temp store"
}
2021/12/24 03:15:14 Waiting for app: TestDataTypes to get deployed
2021/12/24 03:15:19 Waiting for app: TestDataTypes to get deployed
2021/12/24 03:15:24 Waiting for app: TestDataTypes to get deployed
2021/12/24 03:15:29 Waiting for app: TestDataTypes to get deployed
2021/12/24 03:15:29 App: TestDataTypes got deployed
2021/12/24 03:15:29 Starting bucket ops to source bucket
2021/12/24 03:15:34 Waiting for dst bucket item count to get to: 18 curr count: 0
2021/12/24 03:15:39 Waiting for dst bucket item count to get to: 18 curr count: 0
2021/12/24 03:15:44 Waiting for dst bucket item count to get to: 18 curr count: 8
2021/12/24 03:15:44 src & dst bucket item count matched up. src bucket count: 18 dst bucket count: 18
2021/12/24 03:15:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:15:50 Waiting for app: TestDataTypes to get un-deployed
2021/12/24 03:15:55 Waiting for app: TestDataTypes to get un-deployed
2021/12/24 03:16:00 Waiting for app: TestDataTypes to get un-deployed
2021/12/24 03:16:05 Waiting for app: TestDataTypes to get un-deployed
2021/12/24 03:16:05 App: TestDataTypes got un-deployed
2021/12/24 03:16:05 No eventing-con process running
2021/12/24 03:16:05 Function: TestDataTypes delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDataTypes deleting in background"
}
2021/12/24 03:16:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:16:17 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/24 03:16:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:16:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:16:28 Waiting for dst bucket item count to get to: 0 curr count: 15
2021/12/24 03:16:33 Waiting for dst bucket item count to get to: 0 curr count: 5
2021/12/24 03:16:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDataTypes (85.19s)
=== RUN   TestEnoentGet
2021/12/24 03:16:33 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:16:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/24 03:16:39 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:16:44 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:16:50 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:16:55 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:16:55 App: TestEnoentGet got deployed
2021/12/24 03:16:55 Starting bucket ops to source bucket
2021/12/24 03:17:00 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:17:05 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 03:17:05 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:17:05 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:17:10 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:17:15 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:17:20 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:17:25 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:17:25 App: TestEnoentGet got un-deployed
2021/12/24 03:17:25 No eventing-con process running
2021/12/24 03:17:25 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/24 03:17:27 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:17:37 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:17:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:17:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:17:49 Waiting for dst bucket item count to get to: 0 curr count: 87
2021/12/24 03:17:54 Waiting for dst bucket item count to get to: 0 curr count: 37
2021/12/24 03:17:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:17:54 Deploying app: TestEnoentGet. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:17:55 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet stored in temp store"
}
2021/12/24 03:18:00 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:18:05 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:18:10 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:18:15 Waiting for app: TestEnoentGet to get deployed
2021/12/24 03:18:15 App: TestEnoentGet got deployed
2021/12/24 03:18:15 Starting bucket ops to source bucket
2021/12/24 03:18:20 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:18:25 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 03:18:25 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:18:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:18:30 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:18:35 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:18:40 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:18:45 Waiting for app: TestEnoentGet to get un-deployed
2021/12/24 03:18:45 App: TestEnoentGet got un-deployed
2021/12/24 03:18:45 No eventing-con process running
2021/12/24 03:18:45 Function: TestEnoentGet delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentGet deleting in background"
}
2021/12/24 03:18:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:18:57 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:18:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:18:59 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:19:09 Waiting for dst bucket item count to get to: 0 curr count: 96
2021/12/24 03:19:14 Waiting for dst bucket item count to get to: 0 curr count: 46
2021/12/24 03:19:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentGet (160.49s)
=== RUN   TestEnoentDelete
2021/12/24 03:19:14 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:19:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/24 03:19:20 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:19:25 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:19:30 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:19:35 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:19:35 App: TestEnoentDelete got deployed
2021/12/24 03:19:35 Starting bucket ops to source bucket
2021/12/24 03:19:40 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:19:40 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:19:40 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:19:45 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:19:50 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:19:55 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:20:00 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:20:01 App: TestEnoentDelete got un-deployed
2021/12/24 03:20:01 No eventing-con process running
2021/12/24 03:20:01 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/24 03:20:02 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:20:12 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:20:17 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:20:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:20:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:20:29 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:20:34 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:20:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:20:34 Deploying app: TestEnoentDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:20:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete stored in temp store"
}
2021/12/24 03:20:40 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:20:45 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:20:50 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:20:55 Waiting for app: TestEnoentDelete to get deployed
2021/12/24 03:20:55 App: TestEnoentDelete got deployed
2021/12/24 03:20:55 Starting bucket ops to source bucket
2021/12/24 03:21:00 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:21:00 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:21:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:21:06 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:21:11 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:21:16 Waiting for app: TestEnoentDelete to get un-deployed
2021/12/24 03:21:16 App: TestEnoentDelete got un-deployed
2021/12/24 03:21:16 No eventing-con process running
2021/12/24 03:21:16 Function: TestEnoentDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEnoentDelete deleting in background"
}
2021/12/24 03:21:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:21:28 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:21:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:21:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:21:39 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:21:44 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 03:21:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEnoentDelete (150.45s)
=== RUN   TestError
2021/12/24 03:21:54 Waiting for app: TestError to get un-deployed
2021/12/24 03:21:54 App: TestError got un-deployed
2021/12/24 03:21:54 No eventing-con process running
2021/12/24 03:21:54 Function: TestError delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestError not found"
}
2021/12/24 03:21:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:22:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:22:03 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:22:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:22:08 Deploying app: TestError. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:22:09 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError stored in temp store"
}
2021/12/24 03:22:14 Waiting for app: TestError to get deployed
2021/12/24 03:22:19 Waiting for app: TestError to get deployed
2021/12/24 03:22:24 Waiting for app: TestError to get deployed
2021/12/24 03:22:29 Waiting for app: TestError to get deployed
2021/12/24 03:22:29 App: TestError got deployed
2021/12/24 03:22:29 Starting bucket ops to source bucket
2021/12/24 03:22:34 Waiting for dst bucket item count to get to: 300 curr count: 0
2021/12/24 03:22:39 Waiting for dst bucket item count to get to: 300 curr count: 144
2021/12/24 03:22:39 src & dst bucket item count matched up. src bucket count: 300 dst bucket count: 300
2021/12/24 03:22:39 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:22:44 Waiting for app: TestError to get un-deployed
2021/12/24 03:22:50 Waiting for app: TestError to get un-deployed
2021/12/24 03:22:55 Waiting for app: TestError to get un-deployed
2021/12/24 03:23:00 Waiting for app: TestError to get un-deployed
2021/12/24 03:23:00 App: TestError got un-deployed
2021/12/24 03:23:00 No eventing-con process running
2021/12/24 03:23:00 Function: TestError delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestError deleting in background"
}
2021/12/24 03:23:01 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:23:11 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:23:16 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:23:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:23:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:23:28 Waiting for dst bucket item count to get to: 0 curr count: 233
2021/12/24 03:23:33 Waiting for dst bucket item count to get to: 0 curr count: 89
2021/12/24 03:23:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestError (108.96s)
=== RUN   TestCRLF
2021/12/24 03:23:43 Waiting for app: TestCRLF to get un-deployed
2021/12/24 03:23:43 App: TestCRLF got un-deployed
2021/12/24 03:23:43 No eventing-con process running
2021/12/24 03:23:43 Function: TestCRLF delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCRLF not found"
}
2021/12/24 03:23:45 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:23:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:23:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:23:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:23:57 Deploying app: TestCRLF. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:23:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF stored in temp store"
}
2021/12/24 03:24:03 Waiting for app: TestCRLF to get deployed
2021/12/24 03:24:08 Waiting for app: TestCRLF to get deployed
2021/12/24 03:24:13 Waiting for app: TestCRLF to get deployed
2021/12/24 03:24:18 Waiting for app: TestCRLF to get deployed
2021/12/24 03:24:18 App: TestCRLF got deployed
2021/12/24 03:24:18 Starting bucket ops to source bucket
2021/12/24 03:24:23 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 03:24:28 Waiting for dst bucket item count to get to: 100 curr count: 52
2021/12/24 03:24:33 Waiting for dst bucket item count to get to: 100 curr count: 52
2021/12/24 03:24:33 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 03:24:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:24:38 Waiting for app: TestCRLF to get un-deployed
2021/12/24 03:24:44 Waiting for app: TestCRLF to get un-deployed
2021/12/24 03:24:49 Waiting for app: TestCRLF to get un-deployed
2021/12/24 03:24:49 App: TestCRLF got un-deployed
2021/12/24 03:24:49 No eventing-con process running
2021/12/24 03:24:49 Function: TestCRLF delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCRLF deleting in background"
}
2021/12/24 03:24:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:00 Waiting for dst bucket item count to get to: 0 curr count: 77
2021/12/24 03:25:05 Waiting for dst bucket item count to get to: 0 curr count: 77
2021/12/24 03:25:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:25:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:17 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 03:25:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCRLF (103.85s)
=== RUN   TestImportExport
2021/12/24 03:25:27 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:25:27 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/24 03:25:27 No eventing-con process running
2021/12/24 03:25:27 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_n1ql_insert_on_update not found"
}
2021/12/24 03:25:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:25:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:25:46 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:25:46 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/24 03:25:46 No eventing-con process running
2021/12/24 03:25:46 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestImportExport_bucket_op_on_update not found"
}
2021/12/24 03:25:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:25:54 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:25:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:25:59 Deploying app: TestImportExport_n1ql_insert_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:26:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update stored in temp store"
}
2021/12/24 03:26:00 Deploying app: TestImportExport_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:26:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update stored in temp store"
}
2021/12/24 03:26:07 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/24 03:26:12 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/24 03:26:17 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/24 03:26:22 Waiting for app: TestImportExport_n1ql_insert_on_update to get deployed
2021/12/24 03:26:22 App: TestImportExport_n1ql_insert_on_update got deployed
2021/12/24 03:26:27 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/24 03:26:32 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/24 03:26:37 Waiting for app: TestImportExport_bucket_op_on_update to get deployed
2021/12/24 03:26:37 App: TestImportExport_bucket_op_on_update got deployed
2021/12/24 03:26:42 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:26:47 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:26:52 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:26:57 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:26:57 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/24 03:26:57 No eventing-con process running
2021/12/24 03:26:57 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/24 03:27:02 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:27:07 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:27:12 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:27:17 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:27:17 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/24 03:27:17 No eventing-con process running
2021/12/24 03:27:17 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/24 03:27:35 Waiting for app: TestImportExport_bucket_op_on_update to get un-deployed
2021/12/24 03:27:35 App: TestImportExport_bucket_op_on_update got un-deployed
2021/12/24 03:27:35 No eventing-con process running
2021/12/24 03:27:35 Function: TestImportExport_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_bucket_op_on_update deleting in background"
}
2021/12/24 03:27:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:27:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:27:43 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:27:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:27:53 Waiting for app: TestImportExport_n1ql_insert_on_update to get un-deployed
2021/12/24 03:27:53 App: TestImportExport_n1ql_insert_on_update got un-deployed
2021/12/24 03:27:53 No eventing-con process running
2021/12/24 03:27:53 Function: TestImportExport_n1ql_insert_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestImportExport_n1ql_insert_on_update deleting in background"
}
2021/12/24 03:27:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:28:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:28:02 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:28:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestImportExport (169.75s)
=== RUN   TestDeployUndeployLoopNonDefaultSettings
2021/12/24 03:28:17 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:28:17 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:28:17 No eventing-con process running
2021/12/24 03:28:17 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopNonDefaultSettings not found"
}
2021/12/24 03:28:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:28:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:28:25 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:28:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:28:30 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:28:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/24 03:28:31 Starting bucket ops to source bucket
2021/12/24 03:28:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:28:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:28:50 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:28:55 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:28:55 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:29:00 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/24 03:29:00 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/24 03:29:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:29:00 Undeploying app: bucket_op_on_update
2021/12/24 03:29:05 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:29:10 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:29:15 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:29:20 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:29:20 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:29:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:29:20 No eventing-con process running
2021/12/24 03:29:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:29:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:30:04 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:30:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/24 03:30:05 Starting bucket ops to source bucket
2021/12/24 03:30:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:30:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:30:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:30:28 Waiting for dst bucket item count to get to: 5000 curr count: 2804
2021/12/24 03:30:33 Waiting for dst bucket item count to get to: 5000 curr count: 2804
2021/12/24 03:30:33 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:30:38 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/24 03:30:38 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/24 03:30:38 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:30:38 Undeploying app: bucket_op_on_update
2021/12/24 03:30:43 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:30:48 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:30:53 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:30:58 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:30:58 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:30:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:30:58 No eventing-con process running
2021/12/24 03:31:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:31:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:31:42 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:31:43 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/24 03:31:43 Starting bucket ops to source bucket
2021/12/24 03:31:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:31:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:32:01 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:32:06 Waiting for dst bucket item count to get to: 5000 curr count: 2332
2021/12/24 03:32:11 Waiting for dst bucket item count to get to: 5000 curr count: 4832
2021/12/24 03:32:11 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:32:16 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/24 03:32:16 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/24 03:32:16 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:32:16 Undeploying app: bucket_op_on_update
2021/12/24 03:32:22 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:32:27 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:32:32 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:32:37 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:32:37 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:32:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:32:37 No eventing-con process running
2021/12/24 03:32:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:32:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:33:21 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:33:22 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/24 03:33:22 Starting bucket ops to source bucket
2021/12/24 03:33:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:33:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:33:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:33:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:33:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:33:50 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/24 03:33:50 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/24 03:33:50 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:33:50 Undeploying app: bucket_op_on_update
2021/12/24 03:33:55 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:34:00 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:34:05 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:34:10 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:34:10 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:34:10 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:34:10 No eventing-con process running
2021/12/24 03:34:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:34:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:34:54 Deploying app: TestDeployUndeployLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:34:55 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings stored in temp store"
}
2021/12/24 03:34:55 Starting bucket ops to source bucket
2021/12/24 03:35:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:35:08 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:35:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:35:18 Waiting for dst bucket item count to get to: 5000 curr count: 3005
2021/12/24 03:35:23 Waiting for dst bucket item count to get to: 5000 curr count: 3005
2021/12/24 03:35:23 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:35:28 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get deployed
2021/12/24 03:35:28 App: TestDeployUndeployLoopNonDefaultSettings got deployed
2021/12/24 03:35:28 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:35:28 Undeploying app: bucket_op_on_update
2021/12/24 03:35:33 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:35:38 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:35:43 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:35:48 Waiting for app: TestDeployUndeployLoopNonDefaultSettings to get un-deployed
2021/12/24 03:35:48 App: TestDeployUndeployLoopNonDefaultSettings got un-deployed
2021/12/24 03:35:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:35:48 No eventing-con process running
2021/12/24 03:35:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:35:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:36:33 Function: TestDeployUndeployLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopNonDefaultSettings deleting in background"
}
--- PASS: TestDeployUndeployLoopNonDefaultSettings (505.70s)
=== RUN   TestOnUpdateN1QLOp
2021/12/24 03:36:43 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/24 03:36:43 App: TestOnUpdateN1QLOp got un-deployed
2021/12/24 03:36:43 No eventing-con process running
2021/12/24 03:36:43 Function: TestOnUpdateN1QLOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateN1QLOp not found"
}
2021/12/24 03:36:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:36:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:36:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:36:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:36:56 Deploying app: TestOnUpdateN1QLOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:36:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp stored in temp store"
}
2021/12/24 03:36:57 Starting bucket ops to source bucket
2021/12/24 03:37:05 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:37:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:37:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:37:20 Waiting for dst bucket item count to get to: 5000 curr count: 223
2021/12/24 03:37:25 Waiting for dst bucket item count to get to: 5000 curr count: 223
2021/12/24 03:37:25 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:37:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:37:30 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/24 03:37:35 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/24 03:37:40 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/24 03:37:45 Waiting for app: TestOnUpdateN1QLOp to get un-deployed
2021/12/24 03:37:45 App: TestOnUpdateN1QLOp got un-deployed
2021/12/24 03:37:45 No eventing-con process running
2021/12/24 03:37:45 Function: TestOnUpdateN1QLOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateN1QLOp deleting in background"
}
2021/12/24 03:37:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:37:57 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:37:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:37:59 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:38:09 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:38:14 Waiting for dst bucket item count to get to: 0 curr count: 2517
2021/12/24 03:38:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateN1QLOp (101.45s)
=== RUN   TestOnUpdateBucketOpDefaultSettings
2021/12/24 03:38:24 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/24 03:38:24 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/24 03:38:24 No eventing-con process running
2021/12/24 03:38:24 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings not found"
}
2021/12/24 03:38:26 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:38:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:38:33 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:38:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:38:38 Deploying app: TestOnUpdateBucketOpDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:38:39 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings stored in temp store"
}
2021/12/24 03:38:39 Starting bucket ops to source bucket
2021/12/24 03:38:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:38:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:38:57 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:39:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:39:07 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/24 03:39:07 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:39:07 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:39:12 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/24 03:39:17 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/24 03:39:22 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/24 03:39:27 Waiting for app: TestOnUpdateBucketOpDefaultSettings to get un-deployed
2021/12/24 03:39:27 App: TestOnUpdateBucketOpDefaultSettings got un-deployed
2021/12/24 03:39:27 No eventing-con process running
2021/12/24 03:39:27 Function: TestOnUpdateBucketOpDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings deleting in background"
}
2021/12/24 03:39:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:39:39 Waiting for dst bucket item count to get to: 0 curr count: 4703
2021/12/24 03:39:44 Waiting for dst bucket item count to get to: 0 curr count: 2203
2021/12/24 03:39:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:39:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:39:56 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:39:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings (101.56s)
=== RUN   TestOnUpdateBucketOpNonDefaultSettings
2021/12/24 03:40:06 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 03:40:06 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/24 03:40:06 No eventing-con process running
2021/12/24 03:40:06 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/24 03:40:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:40:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:40:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:40:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:40:19 Deploying app: TestOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:40:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/24 03:40:20 Starting bucket ops to source bucket
2021/12/24 03:40:28 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:40:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:40:38 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:40:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:40:43 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:40:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:40:48 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 03:40:53 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 03:40:58 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 03:41:03 Waiting for app: TestOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 03:41:04 App: TestOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/24 03:41:04 No eventing-con process running
2021/12/24 03:41:04 Function: TestOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/24 03:41:05 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:41:15 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:41:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:41:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:41:27 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:41:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpNonDefaultSettings (91.57s)
=== RUN   TestOnUpdateBucketOpDefaultSettings10K
2021/12/24 03:41:37 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/24 03:41:37 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/24 03:41:37 No eventing-con process running
2021/12/24 03:41:37 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings10K not found"
}
2021/12/24 03:41:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:41:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:41:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:41:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:41:51 Deploying app: TestOnUpdateBucketOpDefaultSettings10K. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:41:52 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K stored in temp store"
}
2021/12/24 03:41:52 Starting bucket ops to source bucket
2021/12/24 03:42:00 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:42:05 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:42:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:42:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:42:15 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:42:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:42:20 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/24 03:42:25 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/24 03:42:30 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/24 03:42:35 Waiting for app: TestOnUpdateBucketOpDefaultSettings10K to get un-deployed
2021/12/24 03:42:35 App: TestOnUpdateBucketOpDefaultSettings10K got un-deployed
2021/12/24 03:42:35 No eventing-con process running
2021/12/24 03:42:35 Function: TestOnUpdateBucketOpDefaultSettings10K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings10K deleting in background"
}
2021/12/24 03:42:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:42:47 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:42:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:42:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:42:59 Waiting for dst bucket item count to get to: 0 curr count: 4819
2021/12/24 03:43:04 Waiting for dst bucket item count to get to: 0 curr count: 2319
2021/12/24 03:43:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings10K (96.68s)
=== RUN   TestOnUpdateBucketOpDefaultSettings100K
2021/12/24 03:43:14 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/24 03:43:14 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/24 03:43:14 No eventing-con process running
2021/12/24 03:43:14 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateBucketOpDefaultSettings100K not found"
}
2021/12/24 03:43:16 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:43:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:43:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:43:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:43:27 Deploying app: TestOnUpdateBucketOpDefaultSettings100K. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:43:29 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K stored in temp store"
}
2021/12/24 03:43:29 Starting bucket ops to source bucket
2021/12/24 03:43:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:43:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:43:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:43:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:43:57 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/24 03:43:57 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:43:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:44:02 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/24 03:44:07 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/24 03:44:12 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/24 03:44:17 Waiting for app: TestOnUpdateBucketOpDefaultSettings100K to get un-deployed
2021/12/24 03:44:17 App: TestOnUpdateBucketOpDefaultSettings100K got un-deployed
2021/12/24 03:44:17 No eventing-con process running
2021/12/24 03:44:17 Function: TestOnUpdateBucketOpDefaultSettings100K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateBucketOpDefaultSettings100K deleting in background"
}
2021/12/24 03:44:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:44:29 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:44:34 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/24 03:44:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:44:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:44:46 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:44:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateBucketOpDefaultSettings100K (102.15s)
=== RUN   TestOnDeleteBucketOp
2021/12/24 03:44:56 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/24 03:44:56 App: TestOnDeleteBucketOp got un-deployed
2021/12/24 03:44:56 No eventing-con process running
2021/12/24 03:44:56 Function: TestOnDeleteBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp not found"
}
2021/12/24 03:44:58 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:45:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:45:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:45:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:45:09 Deploying app: TestOnDeleteBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:45:11 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp stored in temp store"
}
2021/12/24 03:45:11 Starting bucket ops to source bucket
2021/12/24 03:45:21 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:45:26 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:45:31 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:45:31 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:45:31 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:45:36 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/24 03:45:42 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/24 03:45:47 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/24 03:45:52 Waiting for app: TestOnDeleteBucketOp to get un-deployed
2021/12/24 03:45:52 App: TestOnDeleteBucketOp got un-deployed
2021/12/24 03:45:52 No eventing-con process running
2021/12/24 03:45:52 Function: TestOnDeleteBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp deleting in background"
}
2021/12/24 03:45:53 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:45:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:46:01 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:46:11 Waiting for dst bucket item count to get to: 0 curr count: 4022
2021/12/24 03:46:16 Waiting for dst bucket item count to get to: 0 curr count: 4022
2021/12/24 03:46:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp (89.79s)
=== RUN   TestOnDeleteBucketOp5K
2021/12/24 03:46:26 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/24 03:46:26 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/24 03:46:26 No eventing-con process running
2021/12/24 03:46:26 Function: TestOnDeleteBucketOp5K delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteBucketOp5K not found"
}
2021/12/24 03:46:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:46:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:46:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:46:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:46:39 Deploying app: TestOnDeleteBucketOp5K. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:46:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K stored in temp store"
}
2021/12/24 03:46:40 Starting bucket ops to source bucket
2021/12/24 03:46:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:46:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:47:01 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:47:01 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:47:01 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:47:06 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/24 03:47:11 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/24 03:47:16 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/24 03:47:21 Waiting for app: TestOnDeleteBucketOp5K to get un-deployed
2021/12/24 03:47:21 App: TestOnDeleteBucketOp5K got un-deployed
2021/12/24 03:47:21 No eventing-con process running
2021/12/24 03:47:22 Function: TestOnDeleteBucketOp5K delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteBucketOp5K deleting in background"
}
2021/12/24 03:47:23 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:47:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:47:31 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:47:41 Waiting for dst bucket item count to get to: 0 curr count: 3633
2021/12/24 03:47:46 Waiting for dst bucket item count to get to: 0 curr count: 3633
2021/12/24 03:47:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteBucketOp5K (89.83s)
=== RUN   TestTimerBucketOp
2021/12/24 03:47:56 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/24 03:47:56 App: TestTimerBucketOp got un-deployed
2021/12/24 03:47:56 No eventing-con process running
2021/12/24 03:47:56 Function: TestTimerBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerBucketOp not found"
}
2021/12/24 03:47:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:48:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:48:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:48:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:48:09 Deploying app: TestTimerBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:48:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp stored in temp store"
}
2021/12/24 03:48:10 Starting bucket ops to source bucket
2021/12/24 03:48:18 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:23 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:28 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:33 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:38 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:48:48 Waiting for dst bucket item count to get to: 5000 curr count: 3105
2021/12/24 03:48:53 Waiting for dst bucket item count to get to: 5000 curr count: 3105
2021/12/24 03:48:53 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:48:53 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:48:58 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/24 03:49:03 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/24 03:49:08 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/24 03:49:13 Waiting for app: TestTimerBucketOp to get un-deployed
2021/12/24 03:49:13 App: TestTimerBucketOp got un-deployed
2021/12/24 03:49:13 No eventing-con process running
2021/12/24 03:49:13 Function: TestTimerBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerBucketOp deleting in background"
}
2021/12/24 03:49:15 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:49:25 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:49:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:49:27 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:49:37 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:49:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerBucketOp (111.27s)
=== RUN   TestTimerInPastBucketOp
2021/12/24 03:49:47 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/24 03:49:47 App: TestTimerInPastBucketOp got un-deployed
2021/12/24 03:49:47 No eventing-con process running
2021/12/24 03:49:47 Function: TestTimerInPastBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerInPastBucketOp not found"
}
2021/12/24 03:49:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:49:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:49:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:50:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:50:00 Deploying app: TestTimerInPastBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:50:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp stored in temp store"
}
2021/12/24 03:50:02 Starting bucket ops to source bucket
2021/12/24 03:50:10 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:15 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:50:40 Waiting for dst bucket item count to get to: 5000 curr count: 1462
2021/12/24 03:50:45 Waiting for dst bucket item count to get to: 5000 curr count: 1462
2021/12/24 03:50:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:50:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:50:50 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/24 03:50:55 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/24 03:51:00 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/24 03:51:05 Waiting for app: TestTimerInPastBucketOp to get un-deployed
2021/12/24 03:51:05 App: TestTimerInPastBucketOp got un-deployed
2021/12/24 03:51:05 No eventing-con process running
2021/12/24 03:51:05 Function: TestTimerInPastBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerInPastBucketOp deleting in background"
}
2021/12/24 03:51:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:51:17 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 03:51:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:51:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:51:28 Waiting for dst bucket item count to get to: 0 curr count: 4133
2021/12/24 03:51:33 Waiting for dst bucket item count to get to: 0 curr count: 1633
2021/12/24 03:51:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerInPastBucketOp (116.54s)
=== RUN   TestDeployUndeployLoopTimer
2021/12/24 03:51:43 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:51:43 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 03:51:43 No eventing-con process running
2021/12/24 03:51:43 Function: TestDeployUndeployLoopTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeployUndeployLoopTimer not found"
}
2021/12/24 03:51:45 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:51:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:51:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:51:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:51:57 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:51:58 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/24 03:51:58 Starting bucket ops to source bucket
2021/12/24 03:52:06 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:11 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:16 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:21 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:26 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:31 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:52:36 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/24 03:52:36 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:52:41 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/24 03:52:41 App: TestDeployUndeployLoopTimer got deployed
2021/12/24 03:52:41 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:52:41 Undeploying app: bucket_op_with_timer
2021/12/24 03:52:46 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:52:51 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:52:56 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:53:01 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:53:01 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 03:53:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:53:01 No eventing-con process running
2021/12/24 03:53:03 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:53:10 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:53:45 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:53:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/24 03:53:46 Starting bucket ops to source bucket
2021/12/24 03:53:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:53:59 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:19 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:24 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:54:24 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:54:29 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/24 03:54:29 App: TestDeployUndeployLoopTimer got deployed
2021/12/24 03:54:29 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:54:30 Undeploying app: bucket_op_with_timer
2021/12/24 03:54:35 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:54:40 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:54:45 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:54:50 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:54:50 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 03:54:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:54:50 No eventing-con process running
2021/12/24 03:54:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:54:59 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:55:34 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:55:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/24 03:55:35 Starting bucket ops to source bucket
2021/12/24 03:55:43 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:55:48 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:55:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:55:58 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:56:03 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:56:08 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:56:13 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:56:13 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:56:18 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/24 03:56:18 App: TestDeployUndeployLoopTimer got deployed
2021/12/24 03:56:18 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:56:18 Undeploying app: bucket_op_with_timer
2021/12/24 03:56:23 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:56:28 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:56:33 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:56:38 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:56:38 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 03:56:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:56:38 No eventing-con process running
2021/12/24 03:56:40 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:56:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:57:22 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:57:23 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/24 03:57:23 Starting bucket ops to source bucket
2021/12/24 03:57:31 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:57:36 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:57:41 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:57:46 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:57:51 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:57:56 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:58:01 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:58:06 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/24 03:58:06 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 03:58:11 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/24 03:58:11 App: TestDeployUndeployLoopTimer got deployed
2021/12/24 03:58:11 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 03:58:11 Undeploying app: bucket_op_with_timer
2021/12/24 03:58:16 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:58:21 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:58:27 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:58:32 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 03:58:32 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 03:58:32 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 03:58:32 No eventing-con process running
2021/12/24 03:58:33 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:58:41 Context: Bucket flush, response code: 200 dump: 
2021/12/24 03:59:16 Deploying app: TestDeployUndeployLoopTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 03:59:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer stored in temp store"
}
2021/12/24 03:59:17 Starting bucket ops to source bucket
2021/12/24 03:59:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:59:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:59:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:59:40 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:59:45 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 03:59:50 Waiting for dst bucket item count to get to: 5000 curr count: 668
2021/12/24 03:59:55 Waiting for dst bucket item count to get to: 5000 curr count: 668
2021/12/24 03:59:55 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:00:00 Waiting for app: TestDeployUndeployLoopTimer to get deployed
2021/12/24 04:00:00 App: TestDeployUndeployLoopTimer got deployed
2021/12/24 04:00:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:00:00 Undeploying app: bucket_op_with_timer
2021/12/24 04:00:05 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 04:00:10 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 04:00:15 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 04:00:20 Waiting for app: TestDeployUndeployLoopTimer to get un-deployed
2021/12/24 04:00:20 App: TestDeployUndeployLoopTimer got un-deployed
2021/12/24 04:00:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:00:20 No eventing-con process running
2021/12/24 04:00:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:00:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:01:04 Function: TestDeployUndeployLoopTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeployUndeployLoopTimer deleting in background"
}
--- PASS: TestDeployUndeployLoopTimer (570.70s)
=== RUN   TestMultipleHandlers
2021/12/24 04:01:14 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/24 04:01:14 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/24 04:01:14 No eventing-con process running
2021/12/24 04:01:14 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_update not found"
}
2021/12/24 04:01:16 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:01:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:01:23 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:01:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:01:33 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/24 04:01:33 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/24 04:01:33 No eventing-con process running
2021/12/24 04:01:33 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMultipleHandlers_bucket_op_on_delete not found"
}
2021/12/24 04:01:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:01:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:01:41 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:01:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:01:46 Deploying app: TestMultipleHandlers_bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:01:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update stored in temp store"
}
2021/12/24 04:01:47 Deploying app: TestMultipleHandlers_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:01:48 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete stored in temp store"
}
2021/12/24 04:01:53 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/24 04:01:58 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/24 04:02:03 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get deployed
2021/12/24 04:02:03 App: TestMultipleHandlers_bucket_op_on_update got deployed
2021/12/24 04:02:09 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/24 04:02:14 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/24 04:02:19 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/24 04:02:24 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get deployed
2021/12/24 04:02:24 App: TestMultipleHandlers_bucket_op_on_delete got deployed
2021/12/24 04:02:24 Starting bucket ops to source bucket
2021/12/24 04:02:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:02:39 Waiting for dst bucket item count to get to: 5000 curr count: 4276
2021/12/24 04:02:44 Waiting for dst bucket item count to get to: 5000 curr count: 4276
2021/12/24 04:02:44 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:02:44 Starting bucket ops to source bucket
2021/12/24 04:02:58 Waiting for dst bucket item count to get to: 10000 curr count: 5388
2021/12/24 04:03:03 Waiting for dst bucket item count to get to: 10000 curr count: 5388
2021/12/24 04:03:03 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:03:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:03:04 Function: TestMultipleHandlers_bucket_op_on_update update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:03:04 Function: TestMultipleHandlers_bucket_op_on_delete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:03:09 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/24 04:03:14 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/24 04:03:19 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/24 04:03:24 Waiting for app: TestMultipleHandlers_bucket_op_on_update to get un-deployed
2021/12/24 04:03:24 App: TestMultipleHandlers_bucket_op_on_update got un-deployed
2021/12/24 04:03:24 No eventing-con process running
2021/12/24 04:03:24 Function: TestMultipleHandlers_bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_update deleting in background"
}
2021/12/24 04:03:26 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:03:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:03:33 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:03:43 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 04:03:48 Waiting for dst bucket item count to get to: 0 curr count: 4996
2021/12/24 04:03:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:03:53 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/24 04:03:58 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/24 04:04:03 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/24 04:04:08 Waiting for app: TestMultipleHandlers_bucket_op_on_delete to get un-deployed
2021/12/24 04:04:08 App: TestMultipleHandlers_bucket_op_on_delete got un-deployed
2021/12/24 04:04:08 No eventing-con process running
2021/12/24 04:04:08 Function: TestMultipleHandlers_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMultipleHandlers_bucket_op_on_delete deleting in background"
}
2021/12/24 04:04:10 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:04:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:04:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:04:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMultipleHandlers (197.72s)
=== RUN   TestPauseResumeLoopDefaultSettings
2021/12/24 04:04:32 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/24 04:04:32 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/24 04:04:32 No eventing-con process running
2021/12/24 04:04:32 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopDefaultSettings not found"
}
2021/12/24 04:04:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:04:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:04:40 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:04:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:04:45 Deploying app: TestPauseResumeLoopDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:04:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings stored in temp store"
}
2021/12/24 04:04:46 Starting bucket ops to source bucket
2021/12/24 04:04:55 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:05:00 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:05:05 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:05:10 Waiting for dst bucket item count to get to: 5000 curr count: 1583
2021/12/24 04:05:15 Waiting for dst bucket item count to get to: 5000 curr count: 1583
2021/12/24 04:05:15 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:05:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:05:20 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/24 04:05:20 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/24 04:05:20 Pausing app: bucket_op_on_update
2021/12/24 04:05:20 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:05:25 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/24 04:05:25 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/24 04:05:25 Resuming app: bucket_op_on_update
2021/12/24 04:05:26 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:05:26 Starting bucket ops to source bucket
2021/12/24 04:05:34 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:05:39 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:05:44 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:05:44 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:05:44 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:05:49 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/24 04:05:49 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/24 04:05:49 Pausing app: bucket_op_on_update
2021/12/24 04:05:49 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:05:54 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/24 04:05:54 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/24 04:05:54 Resuming app: bucket_op_on_update
2021/12/24 04:05:55 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:05:55 Starting bucket ops to source bucket
2021/12/24 04:06:03 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:06:08 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:06:13 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:06:13 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/24 04:06:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:06:18 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/24 04:06:18 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/24 04:06:18 Pausing app: bucket_op_on_update
2021/12/24 04:06:18 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:06:23 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/24 04:06:23 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/24 04:06:23 Resuming app: bucket_op_on_update
2021/12/24 04:06:25 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:06:25 Starting bucket ops to source bucket
2021/12/24 04:06:33 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:06:38 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:06:43 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:06:43 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/24 04:06:43 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:06:48 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/24 04:06:48 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/24 04:06:48 Pausing app: bucket_op_on_update
2021/12/24 04:06:48 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:06:53 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/24 04:06:53 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/24 04:06:53 Resuming app: bucket_op_on_update
2021/12/24 04:06:54 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:06:54 Starting bucket ops to source bucket
2021/12/24 04:07:02 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:07:07 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:07:12 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:07:17 Waiting for dst bucket item count to get to: 25000 curr count: 22500
2021/12/24 04:07:17 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/24 04:07:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:07:22 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to deployed
2021/12/24 04:07:22 Function: TestPauseResumeLoopDefaultSettings status changed to deployed
2021/12/24 04:07:22 Pausing app: bucket_op_on_update
2021/12/24 04:07:22 Function: TestPauseResumeLoopDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:07:27 Waiting for function: TestPauseResumeLoopDefaultSettings status to change to paused
2021/12/24 04:07:27 Function: TestPauseResumeLoopDefaultSettings status changed to paused
2021/12/24 04:07:32 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/24 04:07:37 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/24 04:07:42 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/24 04:07:47 Waiting for app: TestPauseResumeLoopDefaultSettings to get un-deployed
2021/12/24 04:07:47 App: TestPauseResumeLoopDefaultSettings got un-deployed
2021/12/24 04:07:47 No eventing-con process running
2021/12/24 04:07:47 Function: TestPauseResumeLoopDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopDefaultSettings deleting in background"
}
2021/12/24 04:07:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:07:59 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/24 04:08:04 Waiting for dst bucket item count to get to: 0 curr count: 12500
2021/12/24 04:08:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:08:06 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:08:16 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/24 04:08:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopDefaultSettings (234.00s)
=== RUN   TestPauseResumeLoopNonDefaultSettings
2021/12/24 04:08:26 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/24 04:08:26 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/24 04:08:26 No eventing-con process running
2021/12/24 04:08:26 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeLoopNonDefaultSettings not found"
}
2021/12/24 04:08:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:08:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:08:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:08:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:08:39 Deploying app: TestPauseResumeLoopNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:08:41 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings stored in temp store"
}
2021/12/24 04:08:41 Starting bucket ops to source bucket
2021/12/24 04:08:49 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:08:54 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:08:59 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:09:04 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:09:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:09:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:09:19 Waiting for dst bucket item count to get to: 5000 curr count: 2829
2021/12/24 04:09:24 Waiting for dst bucket item count to get to: 5000 curr count: 2829
2021/12/24 04:09:24 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:09:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:09:29 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/24 04:09:29 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/24 04:09:29 Pausing app: bucket_op_with_timer
2021/12/24 04:09:29 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:09:34 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/24 04:09:34 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/24 04:09:34 Resuming app: bucket_op_with_timer
2021/12/24 04:09:35 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:09:35 Starting bucket ops to source bucket
2021/12/24 04:09:43 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:09:48 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:09:53 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:09:58 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:10:03 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:10:08 Waiting for dst bucket item count to get to: 10000 curr count: 8922
2021/12/24 04:10:13 Waiting for dst bucket item count to get to: 10000 curr count: 8922
2021/12/24 04:10:13 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:10:13 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:10:18 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/24 04:10:18 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/24 04:10:18 Pausing app: bucket_op_with_timer
2021/12/24 04:10:18 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:10:23 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/24 04:10:23 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/24 04:10:23 Resuming app: bucket_op_with_timer
2021/12/24 04:10:24 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:10:24 Starting bucket ops to source bucket
2021/12/24 04:10:32 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:10:37 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:10:42 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:10:47 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:10:52 Waiting for dst bucket item count to get to: 15000 curr count: 10000
2021/12/24 04:10:57 Waiting for dst bucket item count to get to: 15000 curr count: 14794
2021/12/24 04:11:02 Waiting for dst bucket item count to get to: 15000 curr count: 14794
2021/12/24 04:11:02 src & dst bucket item count matched up. src bucket count: 15000 dst bucket count: 15000
2021/12/24 04:11:02 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:11:07 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/24 04:11:07 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/24 04:11:07 Pausing app: bucket_op_with_timer
2021/12/24 04:11:08 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:11:13 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/24 04:11:13 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/24 04:11:13 Resuming app: bucket_op_with_timer
2021/12/24 04:11:14 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:11:14 Starting bucket ops to source bucket
2021/12/24 04:11:22 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:27 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:32 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:37 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:42 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:47 Waiting for dst bucket item count to get to: 20000 curr count: 15000
2021/12/24 04:11:47 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/24 04:11:47 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:11:52 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/24 04:11:52 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/24 04:11:52 Pausing app: bucket_op_with_timer
2021/12/24 04:11:52 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:11:57 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/24 04:11:57 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/24 04:11:57 Resuming app: bucket_op_with_timer
2021/12/24 04:11:58 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:11:58 Starting bucket ops to source bucket
2021/12/24 04:12:06 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:12:11 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:12:16 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:12:21 Waiting for dst bucket item count to get to: 25000 curr count: 20000
2021/12/24 04:12:26 Waiting for dst bucket item count to get to: 25000 curr count: 20817
2021/12/24 04:12:31 Waiting for dst bucket item count to get to: 25000 curr count: 21880
2021/12/24 04:12:36 Waiting for dst bucket item count to get to: 25000 curr count: 23563
2021/12/24 04:12:36 src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/24 04:12:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:12:41 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to deployed
2021/12/24 04:12:41 Function: TestPauseResumeLoopNonDefaultSettings status changed to deployed
2021/12/24 04:12:41 Pausing app: bucket_op_with_timer
2021/12/24 04:12:41 Function: TestPauseResumeLoopNonDefaultSettings update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:4 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:77 timer_context_size:15728640 user_prefix:eventing worker_count:4] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:12:46 Waiting for function: TestPauseResumeLoopNonDefaultSettings status to change to paused
2021/12/24 04:12:46 Function: TestPauseResumeLoopNonDefaultSettings status changed to paused
2021/12/24 04:12:51 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/24 04:12:56 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/24 04:13:01 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/24 04:13:06 Waiting for app: TestPauseResumeLoopNonDefaultSettings to get un-deployed
2021/12/24 04:13:06 App: TestPauseResumeLoopNonDefaultSettings got un-deployed
2021/12/24 04:13:06 No eventing-con process running
2021/12/24 04:13:06 Function: TestPauseResumeLoopNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeLoopNonDefaultSettings deleting in background"
}
2021/12/24 04:13:08 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:13:18 Waiting for dst bucket item count to get to: 0 curr count: 18776
2021/12/24 04:13:23 Waiting for dst bucket item count to get to: 0 curr count: 6276
2021/12/24 04:13:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:13:25 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:13:35 Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/24 04:13:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeLoopNonDefaultSettings (319.19s)
=== RUN   TestPauseAndResumeWithWorkerCountChange
2021/12/24 04:13:45 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/24 04:13:45 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/24 04:13:45 No eventing-con process running
2021/12/24 04:13:45 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseAndResumeWithWorkerCountChange not found"
}
2021/12/24 04:13:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:13:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:13:54 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:13:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:13:59 Deploying app: TestPauseAndResumeWithWorkerCountChange. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:14:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange stored in temp store"
}
2021/12/24 04:14:05 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/24 04:14:10 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/24 04:14:15 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/24 04:14:20 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get deployed
2021/12/24 04:14:20 App: TestPauseAndResumeWithWorkerCountChange got deployed
2021/12/24 04:14:20 Starting bucket ops to source bucket
2021/12/24 04:14:29 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:14:34 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:14:39 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:14:44 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:14:49 Waiting for dst bucket item count to get to: 5000 curr count: 2500
2021/12/24 04:14:49 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:14:49 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:14:49 Pausing app: bucket_op_with_timer
2021/12/24 04:14:49 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:14:54 Waiting for function: TestPauseAndResumeWithWorkerCountChange status to change to paused
2021/12/24 04:14:54 Function: TestPauseAndResumeWithWorkerCountChange status changed to paused
2021/12/24 04:14:54 Resuming app: bucket_op_with_timer
2021/12/24 04:14:55 Function: TestPauseAndResumeWithWorkerCountChange update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:14:55 Starting bucket ops to source bucket
2021/12/24 04:15:07 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:15:12 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:15:17 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:15:22 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:15:27 Waiting for dst bucket item count to get to: 10000 curr count: 6903
2021/12/24 04:15:32 Waiting for dst bucket item count to get to: 10000 curr count: 6903
2021/12/24 04:15:32 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:15:37 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/24 04:15:42 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/24 04:15:47 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/24 04:15:52 Waiting for app: TestPauseAndResumeWithWorkerCountChange to get un-deployed
2021/12/24 04:15:52 App: TestPauseAndResumeWithWorkerCountChange got un-deployed
2021/12/24 04:15:52 No eventing-con process running
2021/12/24 04:15:52 Function: TestPauseAndResumeWithWorkerCountChange delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseAndResumeWithWorkerCountChange deleting in background"
}
2021/12/24 04:15:54 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:16:04 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 04:16:09 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:16:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:16:11 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:16:21 Waiting for dst bucket item count to get to: 0 curr count: 9327
2021/12/24 04:16:26 Waiting for dst bucket item count to get to: 0 curr count: 9327
2021/12/24 04:16:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseAndResumeWithWorkerCountChange (170.86s)
=== RUN   TestPauseResumeWithEventingReb
2021/12/24 04:16:36 Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/24 04:16:36 App: TestPauseResumeWithEventingReb got un-deployed
2021/12/24 04:16:36 No eventing-con process running
2021/12/24 04:16:36 Function: TestPauseResumeWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestPauseResumeWithEventingReb not found"
}
2021/12/24 04:16:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:16:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:16:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:16:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:16:49 Deploying app: TestPauseResumeWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:16:51 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb stored in temp store"
}
2021/12/24 04:16:56 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/24 04:17:01 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/24 04:17:06 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/24 04:17:11 Waiting for app: TestPauseResumeWithEventingReb to get deployed
2021/12/24 04:17:11 App: TestPauseResumeWithEventingReb got deployed
2021/12/24 04:17:11 Starting bucket ops to source bucket
2021/12/24 04:17:20 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:17:25 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:17:30 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:17:35 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:17:40 Waiting for dst bucket item count to get to: 5000 curr count: 3453
2021/12/24 04:17:45 Waiting for dst bucket item count to get to: 5000 curr count: 3453
2021/12/24 04:17:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:17:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:17:45 Pausing app: bucket_op_with_timer
2021/12/24 04:17:45 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:17:50 Waiting for function: TestPauseResumeWithEventingReb status to change to paused
2021/12/24 04:17:50 Function: TestPauseResumeWithEventingReb status changed to paused
2021/12/24 04:17:50 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 04:17:50 Removing node(s): [] from the cluster
2021/12/24 04:17:55 Rebalance progress: 29.16666666666666
2021/12/24 04:18:00 Rebalance progress: 29.16666666666666
2021/12/24 04:18:05 Rebalance progress: 100
2021/12/24 04:18:05 Resuming app: bucket_op_with_timer
2021/12/24 04:18:07 Function: TestPauseResumeWithEventingReb update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:true sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:6] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:18:07 Starting bucket ops to source bucket
2021/12/24 04:18:18 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:18:23 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:18:28 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:18:33 Waiting for dst bucket item count to get to: 10000 curr count: 5000
2021/12/24 04:18:38 Waiting for dst bucket item count to get to: 10000 curr count: 5751
2021/12/24 04:18:43 Waiting for dst bucket item count to get to: 10000 curr count: 5751
2021/12/24 04:18:43 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:18:43 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 04:18:48 Rebalance progress: 29.16666666666666
2021/12/24 04:18:53 Rebalance progress: 29.16666666666666
2021/12/24 04:18:58 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/24 04:19:04 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:19:19 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:19:34 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:19:49 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:20:04 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:20:19 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:20:34 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:20:49 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:21:04 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:21:19 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:21:34 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 04:21:49 waitForIndexes waiting for indexes: [{Name:#primary Bucket:default Completion:100} {Name:#primary Bucket:eventing Completion:70} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 172.31.2.134:9300	Node UUID: e74a6e117f9d8f6a4c10f5722d7c907f
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [683-739]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [740-796]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [797-853]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [854-910]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [911-967]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [968-1023]
Producer node: 127.0.0.1:9303	Node UUID: d12819635e95bcccc17dfbf7705331e3
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [342-398]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [399-455]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [456-512]
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [513-569]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [570-626]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 56
	v: [627-682]
Producer node: 127.0.0.1:9302	Node UUID: d12cb4d0c58e41423eeb45efe172ccf4
	workerID: worker_TestPauseResumeWithEventingReb_3
	len: 57
	v: [171-227]
	workerID: worker_TestPauseResumeWithEventingReb_4
	len: 57
	v: [228-284]
	workerID: worker_TestPauseResumeWithEventingReb_5
	len: 57
	v: [285-341]
	workerID: worker_TestPauseResumeWithEventingReb_0
	len: 57
	v: [0-56]
	workerID: worker_TestPauseResumeWithEventingReb_1
	len: 57
	v: [57-113]
	workerID: worker_TestPauseResumeWithEventingReb_2
	len: 57
	v: [114-170]


2021/12/24 04:22:04 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:22:09 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/24 04:22:14 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/24 04:22:19 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/24 04:22:24 clusterSetup.go:397: Waiting for app: TestPauseResumeWithEventingReb to get un-deployed
2021/12/24 04:22:24 clusterSetup.go:405: App: TestPauseResumeWithEventingReb got un-deployed
2021/12/24 04:22:24 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:22:24 eventingSetup.go:418: Function: TestPauseResumeWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestPauseResumeWithEventingReb deleting in background"
}
2021/12/24 04:22:26 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:22:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 04:22:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:22:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:22:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:22:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestPauseResumeWithEventingReb (381.76s)
=== RUN   TestBucketDeleteAfterPause
2021/12/24 04:22:58 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/24 04:22:58 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/24 04:22:58 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:22:58 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteAfterPause not found"
}
2021/12/24 04:22:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:23:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:23:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:23:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:23:11 eventingSetup.go:125: Deploying app: TestBucketDeleteAfterPause. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:23:12 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause stored in temp store"
}
2021/12/24 04:23:17 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/24 04:23:22 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/24 04:23:27 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/24 04:23:32 clusterSetup.go:346: Waiting for app: TestBucketDeleteAfterPause to get deployed
2021/12/24 04:23:32 clusterSetup.go:354: App: TestBucketDeleteAfterPause got deployed
2021/12/24 04:23:32 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:23:42 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:23:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:23:42 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:23:42 handler_test.go:712: Pausing app: bucket_op_on_update
2021/12/24 04:23:42 eventingSetup.go:270: Function: TestBucketDeleteAfterPause update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:23:47 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to paused
2021/12/24 04:23:47 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to paused
2021/12/24 04:23:47 handler_test.go:716: Deleting source bucket: default
2021/12/24 04:23:54 eventingSetup.go:719: Waiting for function: TestBucketDeleteAfterPause status to change to undeployed
2021/12/24 04:23:54 eventingSetup.go:727: Function: TestBucketDeleteAfterPause status changed to undeployed
2021/12/24 04:23:59 clusterSetup.go:397: Waiting for app: TestBucketDeleteAfterPause to get un-deployed
2021/12/24 04:23:59 clusterSetup.go:405: App: TestBucketDeleteAfterPause got un-deployed
2021/12/24 04:23:59 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:23:59 eventingSetup.go:418: Function: TestBucketDeleteAfterPause delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteAfterPause deleting in background"
}
2021/12/24 04:24:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:24:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:24:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:24:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:24:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketDeleteAfterPause (89.26s)
=== RUN   TestChangeFnCodeBetweenPauseResume
2021/12/24 04:24:27 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/24 04:24:27 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/24 04:24:27 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:24:27 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestChangeFnCodeBetweenPauseResume not found"
}
2021/12/24 04:24:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:24:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:24:35 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:24:40 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:24:40 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:24:41 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/24 04:24:46 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/24 04:24:51 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/24 04:24:56 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/24 04:25:01 clusterSetup.go:346: Waiting for app: TestChangeFnCodeBetweenPauseResume to get deployed
2021/12/24 04:25:01 clusterSetup.go:354: App: TestChangeFnCodeBetweenPauseResume got deployed
2021/12/24 04:25:01 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:25:36 handler_test.go:739: Pausing function: TestChangeFnCodeBetweenPauseResume
2021/12/24 04:25:36 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:true description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:25:41 eventingSetup.go:719: Waiting for function: TestChangeFnCodeBetweenPauseResume status to change to paused
2021/12/24 04:25:41 eventingSetup.go:727: Function: TestChangeFnCodeBetweenPauseResume status changed to paused
2021/12/24 04:28:41 handler_test.go:748: Resuming function: TestChangeFnCodeBetweenPauseResume
2021/12/24 04:28:41 eventingSetup.go:125: Deploying app: TestChangeFnCodeBetweenPauseResume. Item count src bucket: 5000 dst bucket: 0
2021/12/24 04:28:43 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume stored in temp store"
}
2021/12/24 04:28:48 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/24 04:28:48 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:28:48 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 0 expected 5000
2021/12/24 04:28:53 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/24 04:28:53 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:28:53 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 2912 expected 5000
2021/12/24 04:28:58 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/24 04:28:58 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:28:58 eventingSetup.go:799: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 3312 expected 5000
2021/12/24 04:29:03 eventingSetup.go:795: Waiting for function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter to get to 5000
2021/12/24 04:29:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:29:03 eventingSetup.go:804: Function: TestChangeFnCodeBetweenPauseResume stat: timer_callback_missing_counter got to 5000
2021/12/24 04:29:03 handler_test.go:753: Undeploying function: TestChangeFnCodeBetweenPauseResume
2021/12/24 04:29:03 eventingSetup.go:270: Function: TestChangeFnCodeBetweenPauseResume update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:29:13 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/24 04:29:18 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/24 04:29:23 clusterSetup.go:397: Waiting for app: TestChangeFnCodeBetweenPauseResume to get un-deployed
2021/12/24 04:29:23 clusterSetup.go:405: App: TestChangeFnCodeBetweenPauseResume got un-deployed
2021/12/24 04:29:23 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:29:23 eventingSetup.go:418: Function: TestChangeFnCodeBetweenPauseResume delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestChangeFnCodeBetweenPauseResume deleting in background"
}
2021/12/24 04:29:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:29:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:29:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/24 04:29:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:29:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:29:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestChangeFnCodeBetweenPauseResume (329.02s)
=== RUN   TestCommentUnCommentOnDelete
2021/12/24 04:29:56 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/24 04:29:56 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/24 04:29:56 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:29:56 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCommentUnCommentOnDelete not found"
}
2021/12/24 04:29:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:30:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:30:04 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:30:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:30:09 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:30:10 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/24 04:30:15 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:30:20 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:30:25 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:30:30 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:30:30 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/24 04:30:30 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:30:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:30:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:30:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:30:44 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:30:44 handler_test.go:780: Undeploying app: TestCommentUnCommentOnDelete
2021/12/24 04:30:44 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:31:14 eventingSetup.go:125: Deploying app: TestCommentUnCommentOnDelete. Item count src bucket: 5000 dst bucket: 5000
2021/12/24 04:31:15 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete stored in temp store"
}
2021/12/24 04:31:20 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:31:25 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:31:30 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:31:36 clusterSetup.go:346: Waiting for app: TestCommentUnCommentOnDelete to get deployed
2021/12/24 04:31:36 clusterSetup.go:354: App: TestCommentUnCommentOnDelete got deployed
2021/12/24 04:31:36 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:31:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:31:53 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:31:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:31:53 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:31:53 handler_test.go:799: Undeploying app: TestCommentUnCommentOnDelete
2021/12/24 04:31:53 eventingSetup.go:270: Function: TestCommentUnCommentOnDelete update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:32:03 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/24 04:32:08 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/24 04:32:13 clusterSetup.go:397: Waiting for app: TestCommentUnCommentOnDelete to get un-deployed
2021/12/24 04:32:13 clusterSetup.go:405: App: TestCommentUnCommentOnDelete got un-deployed
2021/12/24 04:32:13 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:32:13 eventingSetup.go:418: Function: TestCommentUnCommentOnDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCommentUnCommentOnDelete deleting in background"
}
2021/12/24 04:32:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:32:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:32:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:32:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCommentUnCommentOnDelete (160.54s)
=== RUN   TestCPPWorkerCleanup
2021/12/24 04:32:36 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/24 04:32:36 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/24 04:32:36 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:32:36 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCPPWorkerCleanup not found"
}
2021/12/24 04:32:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:32:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:32:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:32:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:32:49 eventingSetup.go:125: Deploying app: TestCPPWorkerCleanup. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:32:51 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup stored in temp store"
}
2021/12/24 04:32:51 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:32:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:33:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:33:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:33:13 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:33:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4146
2021/12/24 04:33:23 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4146
2021/12/24 04:33:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:33:23 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:33:28 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/24 04:33:33 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/24 04:33:38 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/24 04:33:43 clusterSetup.go:397: Waiting for app: TestCPPWorkerCleanup to get un-deployed
2021/12/24 04:33:43 clusterSetup.go:405: App: TestCPPWorkerCleanup got un-deployed
2021/12/24 04:33:43 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:33:43 eventingSetup.go:418: Function: TestCPPWorkerCleanup delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCPPWorkerCleanup deleting in background"
}
2021/12/24 04:33:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:33:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:33:54 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:33:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:34:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:34:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCPPWorkerCleanup (129.60s)
=== RUN   TestWithUserXattrs
2021/12/24 04:34:46 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/24 04:34:46 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/24 04:34:46 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:34:46 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestWithUserXattrs not found"
}
2021/12/24 04:34:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:34:52 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:34:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:34:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:34:59 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:35:01 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:35:03 eventingSetup.go:125: Deploying app: TestWithUserXattrs. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:35:04 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs stored in temp store"
}
2021/12/24 04:35:09 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/24 04:35:14 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/24 04:35:19 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/24 04:35:24 clusterSetup.go:346: Waiting for app: TestWithUserXattrs to get deployed
2021/12/24 04:35:24 clusterSetup.go:354: App: TestWithUserXattrs got deployed
2021/12/24 04:35:24 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:35:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:35:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4128
2021/12/24 04:35:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 4128
2021/12/24 04:35:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:35:44 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:35:49 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/24 04:35:54 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/24 04:35:59 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/24 04:36:04 clusterSetup.go:397: Waiting for app: TestWithUserXattrs to get un-deployed
2021/12/24 04:36:04 clusterSetup.go:405: App: TestWithUserXattrs got un-deployed
2021/12/24 04:36:04 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:36:04 eventingSetup.go:418: Function: TestWithUserXattrs delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestWithUserXattrs deleting in background"
}
2021/12/24 04:36:05 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:36:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:36:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:36:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:36:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:36:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestWithUserXattrs (110.97s)
=== RUN   TestEventProcessingPostBucketFlush
2021/12/24 04:36:37 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:36:37 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/24 04:36:37 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:36:37 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventProcessingPostBucketFlush not found"
}
2021/12/24 04:36:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:36:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:36:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:36:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:36:50 eventingSetup.go:125: Deploying app: TestEventProcessingPostBucketFlush. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:36:51 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush stored in temp store"
}
2021/12/24 04:36:56 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/24 04:37:01 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/24 04:37:06 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/24 04:37:11 clusterSetup.go:346: Waiting for app: TestEventProcessingPostBucketFlush to get deployed
2021/12/24 04:37:11 clusterSetup.go:354: App: TestEventProcessingPostBucketFlush got deployed
2021/12/24 04:37:11 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:37:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:37:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:37:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:37:25 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:37:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:37:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3399
2021/12/24 04:37:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 899
2021/12/24 04:37:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:37:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:37:56 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:37:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:37:56 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:38:22 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 14394
2021/12/24 04:38:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 25000 curr count: 14394
2021/12/24 04:38:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 25000 dst bucket count: 25000
2021/12/24 04:38:27 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:38:27 eventingSetup.go:270: Function: TestEventProcessingPostBucketFlush update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:38:32 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:38:37 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:38:42 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:38:47 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:38:47 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/24 04:38:47 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:38:52 clusterSetup.go:397: Waiting for app: TestEventProcessingPostBucketFlush to get un-deployed
2021/12/24 04:38:52 clusterSetup.go:405: App: TestEventProcessingPostBucketFlush got un-deployed
2021/12/24 04:38:52 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:38:52 eventingSetup.go:418: Function: TestEventProcessingPostBucketFlush delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventProcessingPostBucketFlush deleting in background"
}
2021/12/24 04:38:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:39:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 25000
2021/12/24 04:39:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12500
2021/12/24 04:39:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:39:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:39:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 16512
2021/12/24 04:39:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 16512
2021/12/24 04:39:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventProcessingPostBucketFlush (178.07s)
=== RUN   TestMetaBucketDelete
2021/12/24 04:39:35 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/24 04:39:35 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/24 04:39:35 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:39:35 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDelete not found"
}
2021/12/24 04:39:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:39:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:39:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:39:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:39:48 eventingSetup.go:125: Deploying app: TestMetaBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:39:49 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete stored in temp store"
}
2021/12/24 04:39:54 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/24 04:39:59 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/24 04:40:04 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/24 04:40:09 clusterSetup.go:346: Waiting for app: TestMetaBucketDelete to get deployed
2021/12/24 04:40:09 clusterSetup.go:354: App: TestMetaBucketDelete got deployed
2021/12/24 04:40:09 handler_test.go:902: Deleting metadata bucket: eventing
2021/12/24 04:40:11 handler_test.go:904: Deleted metadata bucket: eventing
2021/12/24 04:40:16 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/24 04:40:16 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/24 04:40:31 clusterSetup.go:397: Waiting for app: TestMetaBucketDelete to get un-deployed
2021/12/24 04:40:31 clusterSetup.go:405: App: TestMetaBucketDelete got un-deployed
2021/12/24 04:40:31 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:40:31 eventingSetup.go:418: Function: TestMetaBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDelete deleting in background"
}
2021/12/24 04:40:32 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:40:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:40:39 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:40:44 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDelete (78.73s)
=== RUN   TestMetaBucketDeleteWithBootstrap
2021/12/24 04:40:54 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:40:54 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:40:54 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:40:54 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaBucketDeleteWithBootstrap not found"
}
2021/12/24 04:40:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:41:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:41:02 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:41:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:41:07 eventingSetup.go:125: Deploying app: TestMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:41:08 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/24 04:41:27 handler_test.go:923: Deleting metadata bucket: eventing
2021/12/24 04:41:27 handler_test.go:925: Deleted metadata bucket: eventing
2021/12/24 04:41:42 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:41:42 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:41:57 clusterSetup.go:397: Waiting for app: TestMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:41:57 clusterSetup.go:405: App: TestMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:41:58 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:41:58 eventingSetup.go:418: Function: TestMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/24 04:41:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:42:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:42:06 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:42:11 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaBucketDeleteWithBootstrap (86.77s)
=== RUN   TestSourceBucketDelete
2021/12/24 04:42:21 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/24 04:42:21 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/24 04:42:21 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:42:21 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDelete not found"
}
2021/12/24 04:42:22 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:42:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:42:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:42:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:42:34 eventingSetup.go:125: Deploying app: TestSourceBucketDelete. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:42:35 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete stored in temp store"
}
2021/12/24 04:42:40 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/24 04:42:45 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/24 04:42:50 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/24 04:42:55 clusterSetup.go:346: Waiting for app: TestSourceBucketDelete to get deployed
2021/12/24 04:42:55 clusterSetup.go:354: App: TestSourceBucketDelete got deployed
2021/12/24 04:42:55 handler_test.go:945: Deleting source bucket: default
2021/12/24 04:42:56 handler_test.go:947: Deleted source bucket: default
2021/12/24 04:43:01 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/24 04:43:02 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/24 04:43:17 clusterSetup.go:397: Waiting for app: TestSourceBucketDelete to get un-deployed
2021/12/24 04:43:17 clusterSetup.go:405: App: TestSourceBucketDelete got un-deployed
2021/12/24 04:43:17 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:43:17 eventingSetup.go:418: Function: TestSourceBucketDelete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDelete deleting in background"
}
2021/12/24 04:43:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:43:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:43:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:43:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDelete (79.13s)
=== RUN   TestSourceBucketDeleteWithBootstrap
2021/12/24 04:43:40 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:43:40 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:43:40 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:43:40 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceBucketDeleteWithBootstrap not found"
}
2021/12/24 04:43:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:43:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:43:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:43:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:43:53 eventingSetup.go:125: Deploying app: TestSourceBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:43:54 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap stored in temp store"
}
2021/12/24 04:44:13 handler_test.go:966: Deleting source bucket: default
2021/12/24 04:44:14 handler_test.go:968: Deleted source bucket: default
2021/12/24 04:44:29 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:44:29 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:44:44 clusterSetup.go:397: Waiting for app: TestSourceBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:44:44 clusterSetup.go:405: App: TestSourceBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:44:44 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:44:44 eventingSetup.go:418: Function: TestSourceBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceBucketDeleteWithBootstrap deleting in background"
}
2021/12/24 04:44:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:44:51 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:44:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:44:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceBucketDeleteWithBootstrap (87.74s)
=== RUN   TestSourceAndMetaBucketDeleteWithBootstrap
2021/12/24 04:45:07 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:45:07 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:45:07 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:45:07 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestSourceAndMetaBucketDeleteWithBootstrap not found"
}
2021/12/24 04:45:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:45:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:45:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:45:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:45:21 eventingSetup.go:125: Deploying app: TestSourceAndMetaBucketDeleteWithBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:45:22 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap stored in temp store"
}
2021/12/24 04:45:40 handler_test.go:988: Deleting source bucket: default
2021/12/24 04:45:42 handler_test.go:990: Deleted source bucket: default
2021/12/24 04:45:42 handler_test.go:992: Deleting metadata bucket: eventing
2021/12/24 04:45:43 handler_test.go:994: Deleted metadata bucket: eventing
2021/12/24 04:45:58 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:45:58 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:46:13 clusterSetup.go:397: Waiting for app: TestSourceAndMetaBucketDeleteWithBootstrap to get un-deployed
2021/12/24 04:46:13 clusterSetup.go:405: App: TestSourceAndMetaBucketDeleteWithBootstrap got un-deployed
2021/12/24 04:46:13 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:46:13 eventingSetup.go:418: Function: TestSourceAndMetaBucketDeleteWithBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestSourceAndMetaBucketDeleteWithBootstrap deleting in background"
}
2021/12/24 04:46:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:46:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:46:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:46:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestSourceAndMetaBucketDeleteWithBootstrap (88.25s)
=== RUN   TestUndeployDuringBootstrap
2021/12/24 04:46:36 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/24 04:46:36 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/24 04:46:36 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:46:36 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployDuringBootstrap not found"
}
2021/12/24 04:46:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:46:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:46:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:46:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:46:49 eventingSetup.go:125: Deploying app: TestUndeployDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:46:50 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap stored in temp store"
}
2021/12/24 04:46:50 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:47:02 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:47:02 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/24 04:47:02 clusterSetup.go:378: App: TestUndeployDuringBootstrap started bootstrap
2021/12/24 04:47:02 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/24 04:47:07 clusterSetup.go:375: Apps undergoing bootstrap: true
2021/12/24 04:47:12 clusterSetup.go:367: No apps undergoing bootstrap
2021/12/24 04:47:17 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/24 04:47:22 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/24 04:47:27 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/24 04:47:27 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/24 04:47:27 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:47:52 clusterSetup.go:397: Waiting for app: TestUndeployDuringBootstrap to get un-deployed
2021/12/24 04:47:52 clusterSetup.go:405: App: TestUndeployDuringBootstrap got un-deployed
2021/12/24 04:47:52 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:47:52 eventingSetup.go:418: Function: TestUndeployDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployDuringBootstrap deleting in background"
}
2021/12/24 04:47:53 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:48:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:48:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/24 04:48:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:48:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:48:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1993
2021/12/24 04:48:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1993
2021/12/24 04:48:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployDuringBootstrap (119.37s)
=== RUN   TestDeleteBeforeUndeploy
2021/12/24 04:48:30 eventingSetup.go:125: Deploying app: TestDeleteBeforeUndeploy. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:48:31 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy stored in temp store"
}
2021/12/24 04:48:36 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/24 04:48:41 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/24 04:48:46 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/24 04:48:51 clusterSetup.go:346: Waiting for app: TestDeleteBeforeUndeploy to get deployed
2021/12/24 04:48:51 clusterSetup.go:354: App: TestDeleteBeforeUndeploy got deployed
2021/12/24 04:48:51 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 422 dump: {
 "name": "ERR_APP_NOT_UNDEPLOYED",
 "code": 22,
 "description": "Function: TestDeleteBeforeUndeploy skipping delete request as it hasn't been undeployed"
}
2021/12/24 04:48:51 eventingSetup.go:270: Function: TestDeleteBeforeUndeploy update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:48:52 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDeleteBeforeUndeploy deleting in background"
}
2021/12/24 04:48:57 clusterSetup.go:397: Waiting for app: TestDeleteBeforeUndeploy to get un-deployed
2021/12/24 04:48:57 clusterSetup.go:405: App: TestDeleteBeforeUndeploy got un-deployed
2021/12/24 04:48:57 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:48:57 eventingSetup.go:418: Function: TestDeleteBeforeUndeploy delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDeleteBeforeUndeploy not found"
}
2021/12/24 04:48:58 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:49:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:49:05 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:49:10 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDeleteBeforeUndeploy (44.56s)
=== RUN   TestUndeployWhenTimersAreFired
2021/12/24 04:49:20 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:49:20 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/24 04:49:20 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:49:20 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWhenTimersAreFired not found"
}
2021/12/24 04:49:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:49:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:49:28 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:49:33 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:49:33 eventingSetup.go:125: Deploying app: TestUndeployWhenTimersAreFired. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:49:34 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired stored in temp store"
}
2021/12/24 04:49:39 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/24 04:49:44 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/24 04:49:49 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/24 04:49:54 clusterSetup.go:346: Waiting for app: TestUndeployWhenTimersAreFired to get deployed
2021/12/24 04:49:54 clusterSetup.go:354: App: TestUndeployWhenTimersAreFired got deployed
2021/12/24 04:49:54 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:50:29 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:50:34 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:50:39 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:50:45 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:50:45 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/24 04:50:45 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:52:30 clusterSetup.go:397: Waiting for app: TestUndeployWhenTimersAreFired to get un-deployed
2021/12/24 04:52:30 clusterSetup.go:405: App: TestUndeployWhenTimersAreFired got un-deployed
2021/12/24 04:52:30 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:52:30 eventingSetup.go:418: Function: TestUndeployWhenTimersAreFired delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWhenTimersAreFired deleting in background"
}
2021/12/24 04:52:31 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:52:41 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:52:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:52:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:52:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:52:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2733
2021/12/24 04:53:03 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 233
2021/12/24 04:53:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWhenTimersAreFired (233.10s)
=== RUN   TestUndeployWithKVFailover
2021/12/24 04:53:13 clusterSetup.go:397: Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/24 04:53:13 clusterSetup.go:405: App: TestUndeployWithKVFailover got un-deployed
2021/12/24 04:53:13 eventingSetup.go:46: No eventing-con process running
2021/12/24 04:53:13 eventingSetup.go:418: Function: TestUndeployWithKVFailover delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployWithKVFailover not found"
}
2021/12/24 04:53:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:53:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:53:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:53:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:53:26 eventingSetup.go:125: Deploying app: TestUndeployWithKVFailover. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:53:27 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover stored in temp store"
}
2021/12/24 04:53:27 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 04:53:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:53:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:53:44 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:53:49 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:53:54 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:53:59 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:54:04 eventingSetup.go:441: Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 04:54:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 04:54:04 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/24 04:54:04 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 04:54:09 Rebalance progress: 29.16666666666666
2021/12/24 04:54:14 Rebalance progress: 29.16666666666666
2021/12/24 04:54:19 Rebalance progress: 100
2021/12/24 04:54:19 Failing over: [127.0.0.1:19003]
2021/12/24 04:54:30 Function: TestUndeployWithKVFailover update settings: map[bucket_cache_age:1000 bucket_cache_size:67108864 cpp_worker_thread_count:1 dcp_stream_boundary:everything deployment_status:false description:Sample app execution_timeout:5 language_compatibility:6.5.0 lcb_inst_capacity:5 log_level:INFO n1ql_consistency:request num_timer_partitions:128 processing_status:false sock_batch_size:1 timer_context_size:15728640 user_prefix:eventing worker_count:3] requested, response code: 0 dump: &{Name: Code:0 Description: Attributes:[] RuntimeInfo: httpResponseCode:0}
2021/12/24 04:55:30 Removing node(s): [] from the cluster
2021/12/24 04:55:35 Rebalance progress: 29.16666666666666
2021/12/24 04:55:40 Rebalance progress: 29.16666666666666
2021/12/24 04:55:45 Rebalance progress: 100
2021/12/24 04:55:50 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/24 04:55:50 App: TestUndeployWithKVFailover got un-deployed
2021/12/24 04:55:50 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:55:55 Waiting for app: TestUndeployWithKVFailover to get un-deployed
2021/12/24 04:55:55 App: TestUndeployWithKVFailover got un-deployed
2021/12/24 04:55:55 No eventing-con process running
2021/12/24 04:55:55 Function: TestUndeployWithKVFailover delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployWithKVFailover deleting in background"
}
2021/12/24 04:55:56 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:56:06 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 04:56:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:56:08 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:56:18 Waiting for dst bucket item count to get to: 0 curr count: 2768
2021/12/24 04:56:23 Waiting for dst bucket item count to get to: 0 curr count: 268
2021/12/24 04:56:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestUndeployWithKVFailover (200.01s)
=== RUN   TestBucketFlushWhileFnDeployed
2021/12/24 04:56:33 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/24 04:56:33 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/24 04:56:33 No eventing-con process running
2021/12/24 04:56:33 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketFlushWhileFnDeployed not found"
}
2021/12/24 04:56:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:56:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:56:41 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:56:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:56:46 Deploying app: TestBucketFlushWhileFnDeployed. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:56:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed stored in temp store"
}
2021/12/24 04:56:52 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/24 04:56:57 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/24 04:57:02 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/24 04:57:07 Waiting for app: TestBucketFlushWhileFnDeployed to get deployed
2021/12/24 04:57:07 App: TestBucketFlushWhileFnDeployed got deployed
2021/12/24 04:57:07 Starting bucket ops to source bucket
2021/12/24 04:57:27 Waiting for dst bucket item count to get to: 20000 curr count: 14521
2021/12/24 04:57:32 Waiting for dst bucket item count to get to: 20000 curr count: 14521
2021/12/24 04:57:32 src & dst bucket item count matched up. src bucket count: 20000 dst bucket count: 20000
2021/12/24 04:57:32 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:57:35 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:57:46 Waiting for dst bucket item count to get to: 0 curr count: 20000
2021/12/24 04:57:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:57:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:57:57 Waiting for dst bucket item count to get to: 0 curr count: 20000
2021/12/24 04:57:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:57:57 Starting bucket ops to source bucket
2021/12/24 04:58:10 Waiting for dst bucket item count to get to: 10000 curr count: 1176
2021/12/24 04:58:15 Waiting for dst bucket item count to get to: 10000 curr count: 6176
2021/12/24 04:58:15 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 04:58:15 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 04:58:20 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/24 04:58:25 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/24 04:58:30 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/24 04:58:35 Waiting for app: TestBucketFlushWhileFnDeployed to get un-deployed
2021/12/24 04:58:35 App: TestBucketFlushWhileFnDeployed got un-deployed
2021/12/24 04:58:35 No eventing-con process running
2021/12/24 04:58:35 Function: TestBucketFlushWhileFnDeployed delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketFlushWhileFnDeployed deleting in background"
}
2021/12/24 04:58:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:58:47 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 04:58:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:58:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:58:58 Waiting for dst bucket item count to get to: 0 curr count: 8614
2021/12/24 04:59:03 Waiting for dst bucket item count to get to: 0 curr count: 3614
2021/12/24 04:59:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBucketFlushWhileFnDeployed (160.69s)
=== RUN   TestTimerOverwrite
2021/12/24 04:59:13 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/24 04:59:13 App: TestTimerOverwrite got un-deployed
2021/12/24 04:59:13 No eventing-con process running
2021/12/24 04:59:13 Function: TestTimerOverwrite delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverwrite not found"
}
2021/12/24 04:59:15 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:59:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:59:21 Context: Bucket flush, response code: 200 dump: 
2021/12/24 04:59:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 04:59:27 Deploying app: TestTimerOverwrite. Item count src bucket: 0 dst bucket: 0
2021/12/24 04:59:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite stored in temp store"
}
2021/12/24 04:59:33 Waiting for app: TestTimerOverwrite to get deployed
2021/12/24 04:59:38 Waiting for app: TestTimerOverwrite to get deployed
2021/12/24 04:59:43 Waiting for app: TestTimerOverwrite to get deployed
2021/12/24 04:59:48 Waiting for app: TestTimerOverwrite to get deployed
2021/12/24 04:59:48 App: TestTimerOverwrite got deployed
2021/12/24 04:59:58 Starting bucket ops to source bucket
2021/12/24 05:00:07 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:12 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:17 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:22 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:27 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:32 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:37 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:42 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:47 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:52 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:00:57 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:01:02 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:01:07 Waiting for dst bucket item count to get to: 5000 curr count: 1153
2021/12/24 05:01:12 Waiting for dst bucket item count to get to: 5000 curr count: 3687
2021/12/24 05:01:12 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 05:01:22 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:01:27 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/24 05:01:32 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/24 05:01:37 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/24 05:01:42 Waiting for app: TestTimerOverwrite to get un-deployed
2021/12/24 05:01:42 App: TestTimerOverwrite got un-deployed
2021/12/24 05:01:42 No eventing-con process running
2021/12/24 05:01:42 Function: TestTimerOverwrite delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverwrite deleting in background"
}
2021/12/24 05:01:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:01:54 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 05:01:59 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/24 05:01:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:02:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:02:10 Waiting for dst bucket item count to get to: 0 curr count: 3815
2021/12/24 05:02:15 Waiting for dst bucket item count to get to: 0 curr count: 3815
2021/12/24 05:02:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverwrite (192.00s)
=== RUN   TestUndeployBackdoorDuringBootstrap
2021/12/24 05:02:20 Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/24 05:02:21 Removing node(s): [] from the cluster
2021/12/24 05:02:26 Rebalance progress: 100
2021/12/24 05:02:26 Failing over: [127.0.0.1:19003]
2021/12/24 05:02:31 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/24 05:02:31 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/24 05:02:31 No eventing-con process running
2021/12/24 05:02:31 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/24 05:02:32 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:02:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:02:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:02:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:02:44 Deploying app: TestUndeployBackdoorDuringBootstrap. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:02:45 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap stored in temp store"
}
2021/12/24 05:02:45 Starting bucket ops to source bucket
2021/12/24 05:02:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:02:55 Context: Post to set retry counter, response code: 200 dump: {
 "code": 0
}
2021/12/24 05:02:55 set retry counter, response &{[123 10 32 34 99 111 100 101 34 58 32 48 10 125] }
2021/12/24 05:04:00 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/24 05:04:00 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/24 05:04:00 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:04:01 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestUndeployBackdoorDuringBootstrap deleting in background"
}
2021/12/24 05:04:06 Waiting for app: TestUndeployBackdoorDuringBootstrap to get un-deployed
2021/12/24 05:04:06 App: TestUndeployBackdoorDuringBootstrap got un-deployed
2021/12/24 05:04:06 No eventing-con process running
2021/12/24 05:04:06 Function: TestUndeployBackdoorDuringBootstrap delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestUndeployBackdoorDuringBootstrap not found"
}
2021/12/24 05:04:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:04:17 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 05:04:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:04:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:04:29 Waiting for dst bucket item count to get to: 0 curr count: 2451
2021/12/24 05:04:34 Waiting for dst bucket item count to get to: 0 curr count: 785
2021/12/24 05:04:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:04:34 Removing node(s): [] from the cluster
2021/12/24 05:04:39 Rebalance progress: 100
--- PASS: TestUndeployBackdoorDuringBootstrap (143.19s)
=== RUN   TestOnUpdateSrcMutation
2021/12/24 05:04:49 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/24 05:04:49 App: TestOnUpdateSrcMutation got un-deployed
2021/12/24 05:04:49 No eventing-con process running
2021/12/24 05:04:49 Function: TestOnUpdateSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutation not found"
}
2021/12/24 05:04:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:04:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:04:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:05:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:05:02 Deploying app: TestOnUpdateSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:05:03 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation stored in temp store"
}
2021/12/24 05:05:03 Starting bucket ops to source bucket
2021/12/24 05:05:25 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 05:05:25 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:05:30 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/24 05:05:35 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/24 05:05:40 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/24 05:05:45 Waiting for app: TestOnUpdateSrcMutation to get un-deployed
2021/12/24 05:05:45 App: TestOnUpdateSrcMutation got un-deployed
2021/12/24 05:05:45 No eventing-con process running
2021/12/24 05:05:45 Function: TestOnUpdateSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutation deleting in background"
}
2021/12/24 05:05:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:05:56 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 05:05:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:05:58 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:06:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutation (84.56s)
=== RUN   TestOnDeleteSrcMutation
2021/12/24 05:06:13 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/24 05:06:13 App: TestOnDeleteSrcMutation got un-deployed
2021/12/24 05:06:13 No eventing-con process running
2021/12/24 05:06:13 Function: TestOnDeleteSrcMutation delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutation not found"
}
2021/12/24 05:06:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:06:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:06:21 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:06:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:06:26 Deploying app: TestOnDeleteSrcMutation. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:06:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation stored in temp store"
}
2021/12/24 05:06:27 Starting bucket ops to source bucket
2021/12/24 05:06:56 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 05:06:56 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:07:01 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/24 05:07:06 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/24 05:07:11 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/24 05:07:16 Waiting for app: TestOnDeleteSrcMutation to get un-deployed
2021/12/24 05:07:16 App: TestOnDeleteSrcMutation got un-deployed
2021/12/24 05:07:16 No eventing-con process running
2021/12/24 05:07:16 Function: TestOnDeleteSrcMutation delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutation deleting in background"
}
2021/12/24 05:07:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:07:28 Waiting for dst bucket item count to get to: 0 curr count: 2500
2021/12/24 05:07:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:07:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:07:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutation (91.09s)
=== RUN   TestOnUpdateSrcMutationWithTimer
2021/12/24 05:07:44 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/24 05:07:44 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/24 05:07:44 No eventing-con process running
2021/12/24 05:07:44 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnUpdateSrcMutationWithTimer not found"
}
2021/12/24 05:07:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:07:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:07:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:07:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:07:57 Deploying app: TestOnUpdateSrcMutationWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:07:59 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer stored in temp store"
}
2021/12/24 05:07:59 Starting bucket ops to source bucket
2021/12/24 05:08:35 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 05:08:36 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:08:41 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/24 05:08:46 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/24 05:08:51 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/24 05:08:56 Waiting for app: TestOnUpdateSrcMutationWithTimer to get un-deployed
2021/12/24 05:08:56 App: TestOnUpdateSrcMutationWithTimer got un-deployed
2021/12/24 05:08:56 No eventing-con process running
2021/12/24 05:08:56 Function: TestOnUpdateSrcMutationWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnUpdateSrcMutationWithTimer deleting in background"
}
2021/12/24 05:08:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:09:07 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 05:09:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:09:09 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:09:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnUpdateSrcMutationWithTimer (99.48s)
=== RUN   TestOnDeleteSrcMutationsWithTimer
2021/12/24 05:09:24 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/24 05:09:24 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/24 05:09:24 No eventing-con process running
2021/12/24 05:09:24 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteSrcMutationsWithTimer not found"
}
2021/12/24 05:09:25 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:09:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:09:32 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:09:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:09:37 Deploying app: TestOnDeleteSrcMutationsWithTimer. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:09:38 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer stored in temp store"
}
2021/12/24 05:09:38 Starting bucket ops to source bucket
2021/12/24 05:10:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 05:10:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:10:22 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/24 05:10:27 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/24 05:10:32 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/24 05:10:37 Waiting for app: TestOnDeleteSrcMutationsWithTimer to get un-deployed
2021/12/24 05:10:37 App: TestOnDeleteSrcMutationsWithTimer got un-deployed
2021/12/24 05:10:37 No eventing-con process running
2021/12/24 05:10:37 Function: TestOnDeleteSrcMutationsWithTimer delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteSrcMutationsWithTimer deleting in background"
}
2021/12/24 05:10:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:10:48 Waiting for dst bucket item count to get to: 0 curr count: 4493
2021/12/24 05:10:53 Waiting for dst bucket item count to get to: 0 curr count: 1993
2021/12/24 05:10:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:10:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:11:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteSrcMutationsWithTimer (106.25s)
=== RUN   TestInterHandlerRecursion
2021/12/24 05:11:10 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:11:10 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/24 05:11:10 No eventing-con process running
2021/12/24 05:11:10 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/24 05:11:11 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:11:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:11:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:11:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:11:28 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:11:28 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/24 05:11:28 No eventing-con process running
2021/12/24 05:11:28 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/24 05:11:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:11:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:11:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:11:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:11:41 Deploying app: TestInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:11:42 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/24 05:11:42 response body {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/24 05:11:47 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:11:52 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:11:57 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:12:02 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:12:02 App: TestInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/24 05:12:02 Deploying app: TestInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:12:02 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
}
2021/12/24 05:12:02 response body {
 "name": "ERR_INTER_FUNCTION_RECURSION",
 "code": 50,
 "description": "Inter handler recursion error"
} err 
2021/12/24 05:12:08 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:12:13 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:12:18 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:12:23 Waiting for app: TestInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:12:23 App: TestInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/24 05:12:23 No eventing-con process running
2021/12/24 05:12:23 Function: TestInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/24 05:12:24 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:12:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:12:31 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:12:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:12:41 Waiting for app: TestInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:12:41 App: TestInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/24 05:12:41 No eventing-con process running
2021/12/24 05:12:41 Function: TestInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/24 05:12:42 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:12:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:12:49 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:12:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterHandlerRecursion (113.91s)
=== RUN   TestInterBucketRecursion
2021/12/24 05:12:59 Deploying app: TestInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:13:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
}
2021/12/24 05:13:00 response body {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/24 05:13:00 Deploying app: TestInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:13:00 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion_function2 causes a cycle involving functions: [TestInterBucketRecursion_function1], hence deployment is disallowed"
}
2021/12/24 05:13:05 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/24 05:13:10 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/24 05:13:15 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/24 05:13:20 Waiting for app: TestInterBucketRecursion_function1 to get deployed
2021/12/24 05:13:20 App: TestInterBucketRecursion_function1 got deployed
2021/12/24 05:13:25 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:13:30 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:13:35 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:13:40 Waiting for app: TestInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:13:40 App: TestInterBucketRecursion_function1 got un-deployed
2021/12/24 05:13:40 No eventing-con process running
2021/12/24 05:13:40 Function: TestInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion_function1 deleting in background"
}
2021/12/24 05:13:42 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:13:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:13:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:13:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:13:58 Waiting for app: TestInterBucketRecursion_function2 to get un-deployed
2021/12/24 05:13:58 App: TestInterBucketRecursion_function2 got un-deployed
2021/12/24 05:13:58 No eventing-con process running
2021/12/24 05:13:58 Function: TestInterBucketRecursion_function2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion_function2 not found"
}
2021/12/24 05:14:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:14:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:14:06 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:14:11 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion (77.45s)
=== RUN   TestLargeHandler
2021/12/24 05:14:11 Changed force_compress value to false
2021/12/24 05:14:11 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:14:12 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_APPCODE_SIZE",
 "code": 39,
 "description": "Function: TestLargeHandler_function handler Code size is more than 131072. Code Size: 131768"
}
2021/12/24 05:14:12 Changed force_compress value to true
2021/12/24 05:14:12 Deploying app: TestLargeHandler_function. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:14:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function stored in temp store"
}
2021/12/24 05:14:18 Waiting for app: TestLargeHandler_function to get deployed
2021/12/24 05:14:23 Waiting for app: TestLargeHandler_function to get deployed
2021/12/24 05:14:28 Waiting for app: TestLargeHandler_function to get deployed
2021/12/24 05:14:33 Waiting for app: TestLargeHandler_function to get deployed
2021/12/24 05:14:33 App: TestLargeHandler_function got deployed
2021/12/24 05:14:39 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/24 05:14:44 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/24 05:14:49 Waiting for app: TestLargeHandler_function to get un-deployed
2021/12/24 05:14:49 App: TestLargeHandler_function got un-deployed
2021/12/24 05:14:49 No eventing-con process running
2021/12/24 05:14:49 Function: TestLargeHandler_function delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestLargeHandler_function deleting in background"
}
2021/12/24 05:14:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:14:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:14:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:15:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestLargeHandler (50.32s)
=== RUN   TestAllowInterHandlerRecursion
2021/12/24 05:15:12 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:15:12 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/24 05:15:12 No eventing-con process running
2021/12/24 05:15:12 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op not found"
}
2021/12/24 05:15:13 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:15:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:15:20 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:15:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:15:30 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:15:30 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/24 05:15:30 No eventing-con process running
2021/12/24 05:15:30 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete not found"
}
2021/12/24 05:15:31 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:15:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:15:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:15:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:15:43 Deploying app: TestAllowInterHandlerRecursion_source_bucket_update_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:15:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
}
2021/12/24 05:15:44 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op stored in temp store"
} err 
2021/12/24 05:15:49 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:15:54 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:15:59 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:16:04 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get deployed
2021/12/24 05:16:04 App: TestAllowInterHandlerRecursion_source_bucket_update_op got deployed
2021/12/24 05:16:04 Deploying app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:16:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
}
2021/12/24 05:16:05 response body {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete stored in temp store"
} err 
2021/12/24 05:16:10 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/24 05:16:15 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/24 05:16:20 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/24 05:16:25 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get deployed
2021/12/24 05:16:25 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got deployed
2021/12/24 05:16:26 TestAllowInterHandlerRecursion_src_bucket_op_on_delete is deployed
2021/12/24 05:16:31 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:16:36 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:16:41 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:16:46 Waiting for app: TestAllowInterHandlerRecursion_source_bucket_update_op to get un-deployed
2021/12/24 05:16:46 App: TestAllowInterHandlerRecursion_source_bucket_update_op got un-deployed
2021/12/24 05:16:46 No eventing-con process running
2021/12/24 05:16:46 Function: TestAllowInterHandlerRecursion_source_bucket_update_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_source_bucket_update_op deleting in background"
}
2021/12/24 05:16:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:16:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:16:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:17:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:17:05 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:17:10 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:17:15 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:17:20 Waiting for app: TestAllowInterHandlerRecursion_src_bucket_op_on_delete to get un-deployed
2021/12/24 05:17:20 App: TestAllowInterHandlerRecursion_src_bucket_op_on_delete got un-deployed
2021/12/24 05:17:20 No eventing-con process running
2021/12/24 05:17:20 Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterHandlerRecursion_src_bucket_op_on_delete deleting in background"
}
2021/12/24 05:17:21 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:17:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:17:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:17:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterHandlerRecursion (151.39s)
=== RUN   TestAllowInterBucketRecursion
2021/12/24 05:17:38 Deploying app: TestAllowInterBucketRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:17:39 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
}
2021/12/24 05:17:39 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 stored in temp store"
} err 
2021/12/24 05:17:44 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/24 05:17:49 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/24 05:17:54 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/24 05:17:59 Waiting for app: TestAllowInterBucketRecursion_function1 to get deployed
2021/12/24 05:17:59 App: TestAllowInterBucketRecursion_function1 got deployed
2021/12/24 05:17:59 Deploying app: TestAllowInterBucketRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:18:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
}
2021/12/24 05:18:00 response body {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 stored in temp store"
} err 
2021/12/24 05:18:05 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/24 05:18:10 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/24 05:18:15 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/24 05:18:20 Waiting for app: TestAllowInterBucketRecursion_function2 to get deployed
2021/12/24 05:18:20 App: TestAllowInterBucketRecursion_function2 got deployed
2021/12/24 05:18:20 TestAllowInterBucketRecursion_function2 is deployed
2021/12/24 05:18:26 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:18:31 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:18:36 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:18:41 Waiting for app: TestAllowInterBucketRecursion_function1 to get un-deployed
2021/12/24 05:18:41 App: TestAllowInterBucketRecursion_function1 got un-deployed
2021/12/24 05:18:41 No eventing-con process running
2021/12/24 05:18:41 Function: TestAllowInterBucketRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function1 deleting in background"
}
2021/12/24 05:18:42 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:18:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:18:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:18:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:19:00 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/24 05:19:05 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/24 05:19:10 Waiting for app: TestAllowInterBucketRecursion_function2 to get un-deployed
2021/12/24 05:19:10 App: TestAllowInterBucketRecursion_function2 got un-deployed
2021/12/24 05:19:10 No eventing-con process running
2021/12/24 05:19:10 Function: TestAllowInterBucketRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestAllowInterBucketRecursion_function2 deleting in background"
}
2021/12/24 05:19:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:19:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestAllowInterBucketRecursion (110.38s)
=== RUN   TestInterBucketRecursion2
2021/12/24 05:19:30 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:42 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:19:53 Deploying app: TestInterBucketRecursion2_function1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:19:54 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 stored in temp store"
}
2021/12/24 05:19:59 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/24 05:20:04 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/24 05:20:09 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/24 05:20:14 Waiting for app: TestInterBucketRecursion2_function1 to get deployed
2021/12/24 05:20:14 App: TestInterBucketRecursion2_function1 got deployed
2021/12/24 05:20:14 Deploying app: TestInterBucketRecursion2_function2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:20:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 stored in temp store"
}
2021/12/24 05:20:20 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/24 05:20:25 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/24 05:20:30 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/24 05:20:35 Waiting for app: TestInterBucketRecursion2_function2 to get deployed
2021/12/24 05:20:35 App: TestInterBucketRecursion2_function2 got deployed
2021/12/24 05:20:35 Deploying app: TestInterBucketRecursion2_function3. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:20:36 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 stored in temp store"
}
2021/12/24 05:20:41 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/24 05:20:46 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/24 05:20:51 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/24 05:20:56 Waiting for app: TestInterBucketRecursion2_function3 to get deployed
2021/12/24 05:20:56 App: TestInterBucketRecursion2_function3 got deployed
2021/12/24 05:20:56 Deploying app: TestInterBucketRecursion2_function4. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:20:56 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestInterBucketRecursion2_function4 causes a cycle involving functions: [TestInterBucketRecursion2_function1 TestInterBucketRecursion2_function2 TestInterBucketRecursion2_function3], hence deployment is disallowed"
}
2021/12/24 05:21:01 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/24 05:21:07 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/24 05:21:12 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/24 05:21:17 Waiting for app: TestInterBucketRecursion2_function1 to get un-deployed
2021/12/24 05:21:17 App: TestInterBucketRecursion2_function1 got un-deployed
2021/12/24 05:21:17 No eventing-con process running
2021/12/24 05:21:17 Function: TestInterBucketRecursion2_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function1 deleting in background"
}
2021/12/24 05:21:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:21:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:21:25 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:21:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:21:35 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/24 05:21:40 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/24 05:21:45 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/24 05:21:50 Waiting for app: TestInterBucketRecursion2_function2 to get un-deployed
2021/12/24 05:21:50 App: TestInterBucketRecursion2_function2 got un-deployed
2021/12/24 05:21:50 No eventing-con process running
2021/12/24 05:21:50 Function: TestInterBucketRecursion2_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function2 deleting in background"
}
2021/12/24 05:21:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:21:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:21:58 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:22:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:22:08 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/24 05:22:14 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/24 05:22:19 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/24 05:22:24 Waiting for app: TestInterBucketRecursion2_function3 to get un-deployed
2021/12/24 05:22:24 App: TestInterBucketRecursion2_function3 got un-deployed
2021/12/24 05:22:24 No eventing-con process running
2021/12/24 05:22:24 Function: TestInterBucketRecursion2_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestInterBucketRecursion2_function3 deleting in background"
}
2021/12/24 05:22:25 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:22:30 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:22:32 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:22:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:22:42 Waiting for app: TestInterBucketRecursion2_function4 to get un-deployed
2021/12/24 05:22:42 App: TestInterBucketRecursion2_function4 got un-deployed
2021/12/24 05:22:42 No eventing-con process running
2021/12/24 05:22:42 Function: TestInterBucketRecursion2_function4 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestInterBucketRecursion2_function4 not found"
}
2021/12/24 05:22:43 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:22:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:22:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:22:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestInterBucketRecursion2 (213.38s)
=== RUN   TestBucketDeleteWithRebOut
2021/12/24 05:23:02 Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/24 05:23:07 Rebalance progress: 100

VB distribution across eventing nodes and their workers::

2021/12/24 05:23:10 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:23:25 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:23:40 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:23:55 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:24:10 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:24:25 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:24:40 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:24:55 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:25:10 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:25:25 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:25:40 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]
2021/12/24 05:25:55 waitForIndexes waiting for indexes: [{Name:#primary Bucket:eventing Completion:71} {Name:#primary Bucket:hello-world Completion:100}]

DCP Stream statuses:

vbucket curr owner:


2021/12/24 05:26:10 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:26:10 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 05:26:10 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 05:26:15 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/24 05:26:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:26:15 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 05:26:16 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 05:26:21 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:

vbucket curr owner:


2021/12/24 05:26:21 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:26:26 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/24 05:26:26 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/24 05:26:26 eventingSetup.go:46: No eventing-con process running
2021/12/24 05:26:26 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestBucketDeleteWithRebOut not found"
}
2021/12/24 05:26:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:26:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:26:34 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:26:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:26:44 eventingSetup.go:125: Deploying app: TestBucketDeleteWithRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:26:45 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut stored in temp store"
}
2021/12/24 05:26:50 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/24 05:26:55 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/24 05:27:00 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/24 05:27:05 clusterSetup.go:346: Waiting for app: TestBucketDeleteWithRebOut to get deployed
2021/12/24 05:27:05 clusterSetup.go:354: App: TestBucketDeleteWithRebOut got deployed

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 127.0.0.1:9303	Node UUID: d12819635e95bcccc17dfbf7705331e3
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]
Producer node: 127.0.0.1:9302	Node UUID: d12cb4d0c58e41423eeb45efe172ccf4
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]
Producer node: 172.31.2.134:9300	Node UUID: e74a6e117f9d8f6a4c10f5722d7c907f
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]


2021/12/24 05:27:05 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:27:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 05:27:10 Rebalance progress: 29.16666666666666
2021/12/24 05:27:15 Rebalance progress: 29.16666666666666
2021/12/24 05:27:20 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 1024
	vb list dump: [0-1023]

vbucket curr owner:
Producer node: 172.31.2.134:9300	Node UUID: e74a6e117f9d8f6a4c10f5722d7c907f
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [911-1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [683-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [797-910]
Producer node: 127.0.0.1:9303	Node UUID: d12819635e95bcccc17dfbf7705331e3
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [456-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 113
	v: [570-682]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [342-455]
Producer node: 127.0.0.1:9302	Node UUID: d12cb4d0c58e41423eeb45efe172ccf4
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 114
	v: [0-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 114
	v: [114-227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 114
	v: [228-341]


2021/12/24 05:27:20 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:27:20 handler_test.go:1674: Deleting source bucket: default
2021/12/24 05:27:22 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 05:27:23 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 05:27:28 Rebalance progress: 29.16666666666666
2021/12/24 05:27:33 Rebalance progress: 29.16666666666666
2021/12/24 05:27:38 Rebalance progress: 100

VB distribution across eventing nodes and their workers::


DCP Stream statuses:
	stream status: running
	len: 901
	vb list dump: [0-8, 12-22, 24-30, 32-48, 50-54, 56-58, 60-64, 66-71, 73-80, 82-96, 98-104, 106-109, 111-117, 119-125, 127-141, 143-144, 146, 148-152, 154-166, 168-170, 172-182, 184-194, 196, 198-216, 218-219, 221-225, 227, 229-230, 232-233, 235-236, 238-250, 252-260, 262, 264-266, 268-272, 274-304, 306, 308-318, 320-326, 328-331, 333-334, 336-338, 340-348, 350-353, 355-364, 366-368, 370-371, 373-381, 383-394, 397-400, 402-405, 407-431, 434-449, 451, 453, 455-458, 461-468, 470-471, 473-485, 487-491, 494-496, 498, 501, 504-505, 507-573, 576-580, 582-596, 598, 600-613, 615-622, 624-642, 645-648, 650-661, 664-686, 688-689, 691-713, 715-718, 720-738, 740-743, 745-757, 759-769, 771-772, 774-780, 782, 784, 787-796, 798-806, 808-820, 822-833, 835, 837-849, 851-855, 857-867, 869, 871-877, 880-886, 888-893, 895-896, 898-930, 932-943, 945-946, 948-967, 969-980, 982-987, 989-992, 994-999, 1001, 1003-1011, 1013, 1015-1021, 1023]
	stream status: stopped
	len: 123
	vb list dump: [9-11, 23, 31, 49, 55, 59, 65, 72, 81, 97, 105, 110, 118, 126, 142, 145, 147, 153, 167, 171, 183, 195, 197, 217, 220, 226, 228, 231, 234, 237, 251, 261, 263, 267, 273, 305, 307, 319, 327, 332, 335, 339, 349, 354, 365, 369, 372, 382, 395-396, 401, 406, 432-433, 450, 452, 454, 459-460, 469, 472, 486, 492-493, 497, 499-500, 502-503, 506, 574-575, 581, 597, 599, 614, 623, 643-644, 649, 662-663, 687, 690, 714, 719, 739, 744, 758, 770, 773, 781, 783, 785-786, 797, 807, 821, 834, 836, 850, 856, 868, 870, 878-879, 887, 894, 897, 931, 944, 947, 968, 981, 988, 993, 1000, 1002, 1012, 1014, 1022]

vbucket curr owner:
Producer node: 127.0.0.1:9302	Node UUID: d12cb4d0c58e41423eeb45efe172ccf4
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 100
	v: [0-8, 12-22, 24-30, 32-48, 50-54, 56-58, 60-64, 66-71, 73-80, 82-96, 98-104, 106-109, 111-113]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 100
	v: [114-117, 119-125, 127-141, 143-144, 146, 148-152, 154-166, 168-170, 172-182, 184-194, 196, 198-216, 218-219, 221-225, 227]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 98
	v: [229-230, 232-233, 235-236, 238-250, 252-260, 262, 264-266, 268-272, 274-304, 306, 308-318, 320-326, 328-331, 333-334, 336-338, 340-341]
Producer node: 172.31.2.134:9300	Node UUID: e74a6e117f9d8f6a4c10f5722d7c907f
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 101
	v: [911-930, 932-943, 945-946, 948-967, 969-980, 982-987, 989-992, 994-999, 1001, 1003-1011, 1013, 1015-1021, 1023]
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 101
	v: [683-686, 688-689, 691-713, 715-718, 720-738, 740-743, 745-757, 759-769, 771-772, 774-780, 782, 784, 787-796]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 100
	v: [798-806, 808-820, 822-833, 835, 837-849, 851-855, 857-867, 869, 871-877, 880-886, 888-893, 895-896, 898-910]
Producer node: 127.0.0.1:9303	Node UUID: d12819635e95bcccc17dfbf7705331e3
	workerID: worker_TestBucketDeleteWithRebOut_0
	len: 99
	v: [342-348, 350-353, 355-364, 366-368, 370-371, 373-381, 383-394, 397-400, 402-405, 407-431, 434-449, 451, 453, 455]
	workerID: worker_TestBucketDeleteWithRebOut_1
	len: 101
	v: [456-458, 461-468, 470-471, 473-485, 487-491, 494-496, 498, 501, 504-505, 507-569]
	workerID: worker_TestBucketDeleteWithRebOut_2
	len: 101
	v: [570-573, 576-580, 582-596, 598, 600-613, 615-622, 624-642, 645-648, 650-661, 664-682]


2021/12/24 05:27:38 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:27:43 eventingSetup.go:719: Waiting for function: TestBucketDeleteWithRebOut status to change to undeployed
2021/12/24 05:27:43 eventingSetup.go:727: Function: TestBucketDeleteWithRebOut status changed to undeployed
2021/12/24 05:27:48 clusterSetup.go:397: Waiting for app: TestBucketDeleteWithRebOut to get un-deployed
2021/12/24 05:27:48 clusterSetup.go:405: App: TestBucketDeleteWithRebOut got un-deployed
2021/12/24 05:27:48 eventingSetup.go:46: No eventing-con process running
2021/12/24 05:27:48 eventingSetup.go:418: Function: TestBucketDeleteWithRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBucketDeleteWithRebOut deleting in background"
}
2021/12/24 05:27:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:27:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:27:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:28:01 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:28:01 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/24 05:28:02 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 05:28:07 Rebalance progress: 100
--- PASS: TestBucketDeleteWithRebOut (312.54s)
=== RUN   TestN1QLRecursion
2021/12/24 05:28:09 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 05:28:11 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/24 05:28:11 Removing node(s): [] from the cluster
2021/12/24 05:28:16 Rebalance progress: 100
2021/12/24 05:28:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:28:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:28:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:28:39 Deploying app: TestN1QLRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:28:40 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
}
2021/12/24 05:28:40 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 stored in temp store"
} err 
2021/12/24 05:28:45 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/24 05:28:50 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/24 05:28:55 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/24 05:29:00 Waiting for app: TestN1QLRecursion_function1 to get deployed
2021/12/24 05:29:00 App: TestN1QLRecursion_function1 got deployed
2021/12/24 05:29:00 TestN1QLRecursion_function1 is deployed
2021/12/24 05:29:00 Deploying app: TestN1QLRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:29:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
}
2021/12/24 05:29:01 response body {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 stored in temp store"
} err 
2021/12/24 05:29:06 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/24 05:29:11 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/24 05:29:16 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/24 05:29:21 Waiting for app: TestN1QLRecursion_function3 to get deployed
2021/12/24 05:29:21 App: TestN1QLRecursion_function3 got deployed
2021/12/24 05:29:21 TestN1QLRecursion_function3 is deployed
2021/12/24 05:29:21 Deploying app: TestN1QLRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:29:21 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
}
2021/12/24 05:29:21 response body {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Inter bucket recursion error; function: TestN1QLRecursion_function2 causes a cycle involving functions: [TestN1QLRecursion_function3 TestN1QLRecursion_function1], hence deployment is disallowed"
} err 
2021/12/24 05:29:21 Success: TestN1QLRecursion_function2 is not deployed
2021/12/24 05:29:26 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/24 05:29:32 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/24 05:29:37 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/24 05:29:42 Waiting for app: TestN1QLRecursion_function1 to get un-deployed
2021/12/24 05:29:42 App: TestN1QLRecursion_function1 got un-deployed
2021/12/24 05:29:42 No eventing-con process running
2021/12/24 05:29:42 Function: TestN1QLRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function1 deleting in background"
}
2021/12/24 05:29:43 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:29:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:29:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:29:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:30:00 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/24 05:30:06 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/24 05:30:11 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/24 05:30:16 Waiting for app: TestN1QLRecursion_function3 to get un-deployed
2021/12/24 05:30:16 App: TestN1QLRecursion_function3 got un-deployed
2021/12/24 05:30:16 No eventing-con process running
2021/12/24 05:30:16 Function: TestN1QLRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLRecursion_function3 deleting in background"
}
2021/12/24 05:30:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:30:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:30:24 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:30:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:30:31 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 05:30:36 Rebalance progress: 100
--- PASS: TestN1QLRecursion (146.40s)
=== RUN   TestN1QLAllowRecursion
2021/12/24 05:30:36 Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 05:30:36 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/24 05:30:36 Removing node(s): [] from the cluster
2021/12/24 05:30:41 Rebalance progress: 100
2021/12/24 05:30:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:30:53 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:30:58 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:31:03 Deploying app: TestN1QLAllowRecursion_function1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:31:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
}
2021/12/24 05:31:04 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 stored in temp store"
} err 
2021/12/24 05:31:09 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/24 05:31:14 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/24 05:31:20 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/24 05:31:25 Waiting for app: TestN1QLAllowRecursion_function1 to get deployed
2021/12/24 05:31:25 App: TestN1QLAllowRecursion_function1 got deployed
2021/12/24 05:31:25 TestN1QLAllowRecursion_function1 is deployed
2021/12/24 05:31:25 Deploying app: TestN1QLAllowRecursion_function3. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:31:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
}
2021/12/24 05:31:26 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 stored in temp store"
} err 
2021/12/24 05:31:31 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/24 05:31:36 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/24 05:31:41 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/24 05:31:46 Waiting for app: TestN1QLAllowRecursion_function3 to get deployed
2021/12/24 05:31:46 App: TestN1QLAllowRecursion_function3 got deployed
2021/12/24 05:31:46 TestN1QLAllowRecursion_function3 is deployed
2021/12/24 05:31:46 Deploying app: TestN1QLAllowRecursion_function2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:31:47 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
}
2021/12/24 05:31:47 response body {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 stored in temp store"
} err 
2021/12/24 05:31:52 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/24 05:31:57 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/24 05:32:02 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/24 05:32:07 Waiting for app: TestN1QLAllowRecursion_function2 to get deployed
2021/12/24 05:32:07 App: TestN1QLAllowRecursion_function2 got deployed
2021/12/24 05:32:07 Success: TestN1QLAllowRecursion_function2 is deployed after allow_interbucket_recursion
2021/12/24 05:32:12 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/24 05:32:17 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/24 05:32:22 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/24 05:32:27 Waiting for app: TestN1QLAllowRecursion_function1 to get un-deployed
2021/12/24 05:32:27 App: TestN1QLAllowRecursion_function1 got un-deployed
2021/12/24 05:32:27 No eventing-con process running
2021/12/24 05:32:27 Function: TestN1QLAllowRecursion_function1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function1 deleting in background"
}
2021/12/24 05:32:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:32:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:32:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:32:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:32:46 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/24 05:32:51 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/24 05:32:56 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/24 05:33:01 Waiting for app: TestN1QLAllowRecursion_function2 to get un-deployed
2021/12/24 05:33:01 App: TestN1QLAllowRecursion_function2 got un-deployed
2021/12/24 05:33:01 No eventing-con process running
2021/12/24 05:33:01 Function: TestN1QLAllowRecursion_function2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function2 deleting in background"
}
2021/12/24 05:33:03 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:33:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:33:10 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:33:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:33:20 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/24 05:33:25 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/24 05:33:30 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/24 05:33:35 Waiting for app: TestN1QLAllowRecursion_function3 to get un-deployed
2021/12/24 05:33:35 App: TestN1QLAllowRecursion_function3 got un-deployed
2021/12/24 05:33:35 No eventing-con process running
2021/12/24 05:33:35 Function: TestN1QLAllowRecursion_function3 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAllowRecursion_function3 deleting in background"
}
2021/12/24 05:33:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:33:42 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:33:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:33:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:33:50 Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 05:33:55 Rebalance progress: 100
--- PASS: TestN1QLAllowRecursion (199.71s)
=== RUN   TestOnDeleteExpiryBucketOp
2021/12/24 05:33:55 Sleeping for some time
2021/12/24 05:34:05 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/24 05:34:05 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/24 05:34:05 No eventing-con process running
2021/12/24 05:34:05 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestOnDeleteExpiryBucketOp not found"
}
2021/12/24 05:34:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:34:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:34:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:34:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:34:19 Deploying app: TestOnDeleteExpiryBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:34:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp stored in temp store"
}
2021/12/24 05:34:20 Starting bucket ops to source bucket
2021/12/24 05:35:09 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:35:14 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 05:35:14 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 05:35:14 Starting bucket ops to source bucket
2021/12/24 05:35:16 Starting bucket ops to source bucket
2021/12/24 05:36:21 Waiting for dst bucket item count to get to: 7000 curr count: 5000
2021/12/24 05:36:21 src & dst bucket item count matched up. src bucket count: 7000 dst bucket count: 7000
2021/12/24 05:36:21 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:36:27 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/24 05:36:32 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/24 05:36:37 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/24 05:36:42 Waiting for app: TestOnDeleteExpiryBucketOp to get un-deployed
2021/12/24 05:36:42 App: TestOnDeleteExpiryBucketOp got un-deployed
2021/12/24 05:36:42 No eventing-con process running
2021/12/24 05:36:42 Function: TestOnDeleteExpiryBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestOnDeleteExpiryBucketOp deleting in background"
}
2021/12/24 05:36:43 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:36:53 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:36:58 Waiting for dst bucket item count to get to: 0 curr count: 501
2021/12/24 05:36:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:37:00 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:37:10 Waiting for dst bucket item count to get to: 0 curr count: 5119
2021/12/24 05:37:15 Waiting for dst bucket item count to get to: 0 curr count: 5119
2021/12/24 05:37:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestOnDeleteExpiryBucketOp (199.79s)
=== RUN   TestTimerOverWriteSameReference
2021/12/24 05:37:20 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/24 05:37:20 App: TestTimerOverWriteSameReference got un-deployed
2021/12/24 05:37:20 No eventing-con process running
2021/12/24 05:37:20 Function: TestTimerOverWriteSameReference delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestTimerOverWriteSameReference not found"
}
2021/12/24 05:37:22 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:37:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:37:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:37:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:37:34 Deploying app: TestTimerOverWriteSameReference. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:37:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference stored in temp store"
}
2021/12/24 05:37:40 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/24 05:37:45 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/24 05:37:50 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/24 05:37:55 Waiting for app: TestTimerOverWriteSameReference to get deployed
2021/12/24 05:37:55 App: TestTimerOverWriteSameReference got deployed
2021/12/24 05:38:05 Starting bucket ops to source bucket
2021/12/24 05:39:06 src & dst bucket item count matched up. src bucket count: 1 dst bucket count: 1
2021/12/24 05:39:16 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:39:21 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/24 05:39:27 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/24 05:39:32 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/24 05:39:37 Waiting for app: TestTimerOverWriteSameReference to get un-deployed
2021/12/24 05:39:37 App: TestTimerOverWriteSameReference got un-deployed
2021/12/24 05:39:37 No eventing-con process running
2021/12/24 05:39:37 Function: TestTimerOverWriteSameReference delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestTimerOverWriteSameReference deleting in background"
}
2021/12/24 05:39:38 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:39:48 Waiting for dst bucket item count to get to: 0 curr count: 845
2021/12/24 05:39:53 Waiting for dst bucket item count to get to: 0 curr count: 345
2021/12/24 05:39:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:39:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:40:05 Waiting for dst bucket item count to get to: 0 curr count: 1
2021/12/24 05:40:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestTimerOverWriteSameReference (170.05s)
=== RUN   TestCancelTimerBucketop
2021/12/24 05:40:10 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/24 05:40:10 App: TestCancelTimerBucketop got un-deployed
2021/12/24 05:40:10 No eventing-con process running
2021/12/24 05:40:10 Function: TestCancelTimerBucketop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCancelTimerBucketop not found"
}
2021/12/24 05:40:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:40:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:40:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:40:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:40:24 Deploying app: TestCancelTimerBucketop. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:40:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop stored in temp store"
}
2021/12/24 05:40:30 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/24 05:40:35 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/24 05:40:40 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/24 05:40:45 Waiting for app: TestCancelTimerBucketop to get deployed
2021/12/24 05:40:45 App: TestCancelTimerBucketop got deployed
2021/12/24 05:40:45 Starting bucket ops to source bucket
2021/12/24 05:40:52 Starting bucket ops to source bucket
2021/12/24 05:41:55 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/24 05:41:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:42:00 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/24 05:42:05 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/24 05:42:10 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/24 05:42:15 Waiting for app: TestCancelTimerBucketop to get un-deployed
2021/12/24 05:42:15 App: TestCancelTimerBucketop got un-deployed
2021/12/24 05:42:15 No eventing-con process running
2021/12/24 05:42:15 Function: TestCancelTimerBucketop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCancelTimerBucketop deleting in background"
}
2021/12/24 05:42:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:42:27 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:42:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:42:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:42:39 Waiting for dst bucket item count to get to: 0 curr count: 937
2021/12/24 05:42:44 Waiting for dst bucket item count to get to: 0 curr count: 437
2021/12/24 05:42:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCancelTimerBucketop (158.46s)
=== RUN   TestJSExpiryDate
2021/12/24 05:42:49 Starting bucket ops to source bucket
2021/12/24 05:43:07 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 05:43:07 Deploying app: TestJSExpiryDate. Item count src bucket: 5000 dst bucket: 0
2021/12/24 05:43:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate stored in temp store"
}
2021/12/24 05:43:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:43:33 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:43:38 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/24 05:43:43 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/24 05:43:48 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/24 05:43:53 Waiting for app: TestJSExpiryDate to get un-deployed
2021/12/24 05:43:53 App: TestJSExpiryDate got un-deployed
2021/12/24 05:43:53 No eventing-con process running
2021/12/24 05:43:53 Function: TestJSExpiryDate delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestJSExpiryDate deleting in background"
}
2021/12/24 05:43:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:44:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:44:03 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:44:08 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestJSExpiryDate (83.81s)
=== RUN   TestBinaryDoc
2021/12/24 05:44:13 Starting bucket ops to source bucket
2021/12/24 05:44:13 Deploying app: TestBinaryDoc. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:44:14 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc stored in temp store"
}
2021/12/24 05:44:14 Starting bucket ops to source bucket
2021/12/24 05:44:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:44:14 Deleting source bucket docs now
2021/12/24 05:44:14 Starting bucket ops to source bucket
2021/12/24 05:44:51 Waiting for dst bucket item count to get to: 32 curr count: 24
2021/12/24 05:44:56 Waiting for dst bucket item count to get to: 32 curr count: 28
2021/12/24 05:44:56 src & dst bucket item count matched up. src bucket count: 32 dst bucket count: 32
2021/12/24 05:44:56 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:45:01 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/24 05:45:06 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/24 05:45:11 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/24 05:45:16 Waiting for app: TestBinaryDoc to get un-deployed
2021/12/24 05:45:16 App: TestBinaryDoc got un-deployed
2021/12/24 05:45:16 No eventing-con process running
2021/12/24 05:45:16 Function: TestBinaryDoc delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestBinaryDoc deleting in background"
}
2021/12/24 05:45:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:45:28 Waiting for dst bucket item count to get to: 0 curr count: 22
2021/12/24 05:45:33 Waiting for dst bucket item count to get to: 0 curr count: 4
2021/12/24 05:45:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:45:35 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:45:45 Waiting for dst bucket item count to get to: 0 curr count: 32
2021/12/24 05:45:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBinaryDoc (97.35s)
=== RUN   TestConstantBindings
2021/12/24 05:45:45 Deploying app: TestConstantBindings. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:45:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings stored in temp store"
}
2021/12/24 05:45:46 Starting bucket ops to source bucket
2021/12/24 05:45:52 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/24 05:45:57 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/24 05:46:02 Waiting for dst bucket item count to get to: 6000 curr count: 0
2021/12/24 05:46:07 Waiting for dst bucket item count to get to: 6000 curr count: 2096
2021/12/24 05:46:12 Waiting for dst bucket item count to get to: 6000 curr count: 2096
2021/12/24 05:46:12 src & dst bucket item count matched up. src bucket count: 6000 dst bucket count: 6000
2021/12/24 05:46:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:46:17 Waiting for app: TestConstantBindings to get un-deployed
2021/12/24 05:46:22 Waiting for app: TestConstantBindings to get un-deployed
2021/12/24 05:46:27 Waiting for app: TestConstantBindings to get un-deployed
2021/12/24 05:46:32 Waiting for app: TestConstantBindings to get un-deployed
2021/12/24 05:46:32 App: TestConstantBindings got un-deployed
2021/12/24 05:46:32 No eventing-con process running
2021/12/24 05:46:32 Function: TestConstantBindings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestConstantBindings deleting in background"
}
2021/12/24 05:46:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:46:44 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:46:49 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/24 05:46:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:46:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:47:01 Waiting for dst bucket item count to get to: 0 curr count: 5056
2021/12/24 05:47:06 Waiting for dst bucket item count to get to: 0 curr count: 5056
2021/12/24 05:47:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestConstantBindings (80.65s)
=== RUN   TestCodeInsight
2021/12/24 05:47:11 Waiting for app: TestCodeInsight to get un-deployed
2021/12/24 05:47:11 App: TestCodeInsight got un-deployed
2021/12/24 05:47:11 No eventing-con process running
2021/12/24 05:47:11 Function: TestCodeInsight delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestCodeInsight not found"
}
2021/12/24 05:47:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:47:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:47:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:47:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:47:24 Starting bucket ops to source bucket
2021/12/24 05:47:25 Deploying app: TestCodeInsight. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:47:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight stored in temp store"
}
2021/12/24 05:47:31 Waiting for app: TestCodeInsight to get deployed
2021/12/24 05:47:36 Waiting for app: TestCodeInsight to get deployed
2021/12/24 05:47:41 Waiting for app: TestCodeInsight to get deployed
2021/12/24 05:47:46 Waiting for app: TestCodeInsight to get deployed
2021/12/24 05:47:46 App: TestCodeInsight got deployed
2021/12/24 05:47:51 Waiting for dst bucket item count to get to: 1000 curr count: 447
2021/12/24 05:47:51 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/24 05:47:56 Waiting for app: TestCodeInsight to get un-deployed
2021/12/24 05:48:01 Waiting for app: TestCodeInsight to get un-deployed
2021/12/24 05:48:06 Waiting for app: TestCodeInsight to get un-deployed
2021/12/24 05:48:11 Waiting for app: TestCodeInsight to get un-deployed
2021/12/24 05:48:11 App: TestCodeInsight got un-deployed
2021/12/24 05:48:11 No eventing-con process running
2021/12/24 05:48:11 Function: TestCodeInsight delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestCodeInsight deleting in background"
}
2021/12/24 05:48:13 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:48:23 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:48:28 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/24 05:48:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:48:30 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:48:40 Waiting for dst bucket item count to get to: 0 curr count: 631
2021/12/24 05:48:45 Waiting for dst bucket item count to get to: 0 curr count: 631
2021/12/24 05:48:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestCodeInsight (99.40s)
=== RUN   TestParserTransform
--- PASS: TestParserTransform (0.04s)
=== RUN   TestParserDetect
--- PASS: TestParserDetect (0.00s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	12742.974s

Functional tests took 3 hours, 32 minutes and 26 seconds.

N1ql tests

2021/12/24 05:48:56 Started n1ql
2021/12/24 05:49:05 Adding node: https://127.0.0.1:19001 with role: kv,index,n1ql to the cluster
2021/12/24 05:49:05 Error in adding nodes : , response: ["Prepare join failed. Node is already part of cluster."]
2021/12/24 05:49:05 Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 05:49:05 Removing node(s): [] from the cluster
2021/12/24 05:49:10 Rebalance progress: 100
=== RUN   TestRecursiveMutationN1QL
2021/12/24 05:49:15 Waiting for app: TestRecursiveMutationN1QL to get un-deployed
2021/12/24 05:49:15 App: TestRecursiveMutationN1QL got un-deployed
2021/12/24 05:49:15 No eventing-con process running
2021/12/24 05:49:15 Function: TestRecursiveMutationN1QL delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestRecursiveMutationN1QL not found"
}
2021/12/24 05:49:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:49:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:49:24 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:49:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:49:29 Deploying app: TestRecursiveMutationN1QL. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:49:29 Context: Post to main store, response code: 400 dump: {
 "name": "ERR_INTER_BUCKET_RECURSION",
 "code": 51,
 "description": "Function: TestRecursiveMutationN1QL N1QL DML to source keyspace default:default"
}
--- PASS: TestRecursiveMutationN1QL (18.58s)
=== RUN   TestFlexReset1
2021/12/24 05:49:34 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/24 05:49:34 App: TestFlexReset1 got un-deployed
2021/12/24 05:49:34 No eventing-con process running
2021/12/24 05:49:34 Function: TestFlexReset1 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset1 not found"
}
2021/12/24 05:49:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:49:41 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:49:42 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:49:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:49:47 Starting bucket ops to source bucket
2021/12/24 05:49:48 Deploying app: TestFlexReset1. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:49:49 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 stored in temp store"
}
2021/12/24 05:49:54 Waiting for app: TestFlexReset1 to get deployed
2021/12/24 05:49:59 Waiting for app: TestFlexReset1 to get deployed
2021/12/24 05:50:04 Waiting for app: TestFlexReset1 to get deployed
2021/12/24 05:50:09 Waiting for app: TestFlexReset1 to get deployed
2021/12/24 05:50:09 App: TestFlexReset1 got deployed
2021/12/24 05:50:14 Waiting for dst bucket item count to get to: 1000 curr count: 500
2021/12/24 05:50:14 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/24 05:50:14 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:50:19 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/24 05:50:25 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/24 05:50:30 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/24 05:50:35 Waiting for app: TestFlexReset1 to get un-deployed
2021/12/24 05:50:35 App: TestFlexReset1 got un-deployed
2021/12/24 05:50:35 No eventing-con process running
2021/12/24 05:50:35 Function: TestFlexReset1 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset1 deleting in background"
}
2021/12/24 05:50:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:50:46 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:50:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:50:48 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:50:58 Waiting for dst bucket item count to get to: 0 curr count: 751
2021/12/24 05:51:03 Waiting for dst bucket item count to get to: 0 curr count: 251
2021/12/24 05:51:03 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset1 (94.53s)
=== RUN   TestFlexReset2
2021/12/24 05:51:08 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/24 05:51:08 App: TestFlexReset2 got un-deployed
2021/12/24 05:51:08 No eventing-con process running
2021/12/24 05:51:08 Function: TestFlexReset2 delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestFlexReset2 not found"
}
2021/12/24 05:51:10 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:51:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:51:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:51:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:51:22 Starting bucket ops to source bucket
2021/12/24 05:51:22 Deploying app: TestFlexReset2. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:51:24 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 stored in temp store"
}
2021/12/24 05:51:29 Waiting for app: TestFlexReset2 to get deployed
2021/12/24 05:51:34 Waiting for app: TestFlexReset2 to get deployed
2021/12/24 05:51:39 Waiting for app: TestFlexReset2 to get deployed
2021/12/24 05:51:44 Waiting for app: TestFlexReset2 to get deployed
2021/12/24 05:51:44 App: TestFlexReset2 got deployed
2021/12/24 05:51:49 Waiting for dst bucket item count to get to: 1000 curr count: 691
2021/12/24 05:51:54 Waiting for dst bucket item count to get to: 1000 curr count: 691
2021/12/24 05:51:54 src & dst bucket item count matched up. src bucket count: 1000 dst bucket count: 1000
2021/12/24 05:51:54 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:51:59 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/24 05:52:04 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/24 05:52:09 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/24 05:52:14 Waiting for app: TestFlexReset2 to get un-deployed
2021/12/24 05:52:14 App: TestFlexReset2 got un-deployed
2021/12/24 05:52:14 No eventing-con process running
2021/12/24 05:52:14 Function: TestFlexReset2 delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestFlexReset2 deleting in background"
}
2021/12/24 05:52:16 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:52:26 Waiting for dst bucket item count to get to: 0 curr count: 1000
2021/12/24 05:52:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:52:28 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:52:38 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/24 05:52:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestFlexReset2 (94.27s)
=== RUN   TestN1QLLabelledBreak
2021/12/24 05:52:43 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/24 05:52:43 App: TestN1QLLabelledBreak got un-deployed
2021/12/24 05:52:43 No eventing-con process running
2021/12/24 05:52:43 Function: TestN1QLLabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLLabelledBreak not found"
}
2021/12/24 05:52:44 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:52:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:52:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:52:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:52:56 Deploying app: TestN1QLLabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:52:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak stored in temp store"
}
2021/12/24 05:53:02 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/24 05:53:07 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/24 05:53:12 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/24 05:53:17 Waiting for app: TestN1QLLabelledBreak to get deployed
2021/12/24 05:53:17 App: TestN1QLLabelledBreak got deployed
2021/12/24 05:53:17 Starting bucket ops to source bucket
2021/12/24 05:53:27 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/24 05:53:32 Waiting for dst bucket item count to get to: 10000 curr count: 130
2021/12/24 05:53:37 Waiting for dst bucket item count to get to: 10000 curr count: 130
2021/12/24 05:53:42 Waiting for dst bucket item count to get to: 10000 curr count: 3265
2021/12/24 05:53:47 Waiting for dst bucket item count to get to: 10000 curr count: 3265
2021/12/24 05:53:52 Waiting for dst bucket item count to get to: 10000 curr count: 7342
2021/12/24 05:53:57 Waiting for dst bucket item count to get to: 10000 curr count: 7342
2021/12/24 05:53:57 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 05:53:57 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:54:02 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/24 05:54:07 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/24 05:54:12 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/24 05:54:17 Waiting for app: TestN1QLLabelledBreak to get un-deployed
2021/12/24 05:54:17 App: TestN1QLLabelledBreak got un-deployed
2021/12/24 05:54:17 No eventing-con process running
2021/12/24 05:54:17 Function: TestN1QLLabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLLabelledBreak deleting in background"
}
2021/12/24 05:54:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:54:29 Waiting for dst bucket item count to get to: 0 curr count: 4949
2021/12/24 05:54:34 Waiting for dst bucket item count to get to: 0 curr count: 2449
2021/12/24 05:54:34 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:54:36 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:54:46 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 05:54:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLLabelledBreak (128.10s)
=== RUN   TestN1QLUnlabelledBreak
2021/12/24 05:54:51 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/24 05:54:51 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/24 05:54:51 No eventing-con process running
2021/12/24 05:54:51 Function: TestN1QLUnlabelledBreak delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLUnlabelledBreak not found"
}
2021/12/24 05:54:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:54:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:54:59 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:55:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:55:04 Deploying app: TestN1QLUnlabelledBreak. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:55:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak stored in temp store"
}
2021/12/24 05:55:10 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/24 05:55:15 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/24 05:55:20 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/24 05:55:25 Waiting for app: TestN1QLUnlabelledBreak to get deployed
2021/12/24 05:55:25 App: TestN1QLUnlabelledBreak got deployed
2021/12/24 05:55:25 Starting bucket ops to source bucket
2021/12/24 05:55:35 Waiting for dst bucket item count to get to: 10000 curr count: 588
2021/12/24 05:55:40 Waiting for dst bucket item count to get to: 10000 curr count: 588
2021/12/24 05:55:45 Waiting for dst bucket item count to get to: 10000 curr count: 588
2021/12/24 05:55:50 Waiting for dst bucket item count to get to: 10000 curr count: 5786
2021/12/24 05:55:55 Waiting for dst bucket item count to get to: 10000 curr count: 5786
2021/12/24 05:55:55 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 05:55:55 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:56:00 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/24 05:56:05 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/24 05:56:10 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/24 05:56:15 Waiting for app: TestN1QLUnlabelledBreak to get un-deployed
2021/12/24 05:56:15 App: TestN1QLUnlabelledBreak got un-deployed
2021/12/24 05:56:15 No eventing-con process running
2021/12/24 05:56:15 Function: TestN1QLUnlabelledBreak delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLUnlabelledBreak deleting in background"
}
2021/12/24 05:56:17 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:56:27 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 05:56:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:56:29 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:56:39 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 05:56:44 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 05:56:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLUnlabelledBreak (118.37s)
=== RUN   TestN1QLThrowStatement
2021/12/24 05:56:49 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/24 05:56:49 App: TestN1QLThrowStatement got un-deployed
2021/12/24 05:56:49 No eventing-con process running
2021/12/24 05:56:49 Function: TestN1QLThrowStatement delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLThrowStatement not found"
}
2021/12/24 05:56:51 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:56:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:56:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:57:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:57:03 Deploying app: TestN1QLThrowStatement. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:57:04 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement stored in temp store"
}
2021/12/24 05:57:09 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/24 05:57:14 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/24 05:57:19 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/24 05:57:24 Waiting for app: TestN1QLThrowStatement to get deployed
2021/12/24 05:57:24 App: TestN1QLThrowStatement got deployed
2021/12/24 05:57:24 Starting bucket ops to source bucket
2021/12/24 05:57:33 Waiting for dst bucket item count to get to: 10000 curr count: 0
2021/12/24 05:57:38 Waiting for dst bucket item count to get to: 10000 curr count: 237
2021/12/24 05:57:43 Waiting for dst bucket item count to get to: 10000 curr count: 756
2021/12/24 05:57:48 Waiting for dst bucket item count to get to: 10000 curr count: 2656
2021/12/24 05:57:53 Waiting for dst bucket item count to get to: 10000 curr count: 4699
2021/12/24 05:57:58 Waiting for dst bucket item count to get to: 10000 curr count: 6741
2021/12/24 05:58:03 Waiting for dst bucket item count to get to: 10000 curr count: 8759
2021/12/24 05:58:08 Waiting for dst bucket item count to get to: 10000 curr count: 9580
2021/12/24 05:58:08 src & dst bucket item count matched up. src bucket count: 10000 dst bucket count: 10000
2021/12/24 05:58:08 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 05:58:13 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/24 05:58:18 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/24 05:58:23 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/24 05:58:28 Waiting for app: TestN1QLThrowStatement to get un-deployed
2021/12/24 05:58:28 App: TestN1QLThrowStatement got un-deployed
2021/12/24 05:58:28 No eventing-con process running
2021/12/24 05:58:28 Function: TestN1QLThrowStatement delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLThrowStatement deleting in background"
}
2021/12/24 05:58:30 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:58:40 Waiting for dst bucket item count to get to: 0 curr count: 3544
2021/12/24 05:58:45 Waiting for dst bucket item count to get to: 0 curr count: 3544
2021/12/24 05:58:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:58:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:58:57 Waiting for dst bucket item count to get to: 0 curr count: 10000
2021/12/24 05:58:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLThrowStatement (133.05s)
=== RUN   TestN1QLNestedForLoop
2021/12/24 05:59:02 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/24 05:59:02 App: TestN1QLNestedForLoop got un-deployed
2021/12/24 05:59:02 No eventing-con process running
2021/12/24 05:59:02 Function: TestN1QLNestedForLoop delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLNestedForLoop not found"
}
2021/12/24 05:59:04 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:59:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:59:11 Context: Bucket flush, response code: 200 dump: 
2021/12/24 05:59:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 05:59:16 Deploying app: TestN1QLNestedForLoop. Item count src bucket: 0 dst bucket: 0
2021/12/24 05:59:17 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop stored in temp store"
}
2021/12/24 05:59:22 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/24 05:59:27 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/24 05:59:32 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/24 05:59:37 Waiting for app: TestN1QLNestedForLoop to get deployed
2021/12/24 05:59:37 App: TestN1QLNestedForLoop got deployed
2021/12/24 05:59:37 Starting bucket ops to source bucket
2021/12/24 05:59:47 Waiting for dst bucket item count to get to: 5000 curr count: 75
2021/12/24 05:59:52 Waiting for dst bucket item count to get to: 5000 curr count: 75
2021/12/24 05:59:57 Waiting for dst bucket item count to get to: 5000 curr count: 1903
2021/12/24 06:00:02 Waiting for dst bucket item count to get to: 5000 curr count: 1903
2021/12/24 06:00:07 Waiting for dst bucket item count to get to: 5000 curr count: 1903
2021/12/24 06:00:12 Waiting for dst bucket item count to get to: 5000 curr count: 4288
2021/12/24 06:00:17 Waiting for dst bucket item count to get to: 5000 curr count: 4288
2021/12/24 06:00:17 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 06:00:17 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:00:22 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/24 06:00:27 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/24 06:00:32 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/24 06:00:37 Waiting for app: TestN1QLNestedForLoop to get un-deployed
2021/12/24 06:00:37 App: TestN1QLNestedForLoop got un-deployed
2021/12/24 06:00:37 No eventing-con process running
2021/12/24 06:00:37 Function: TestN1QLNestedForLoop delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLNestedForLoop deleting in background"
}
2021/12/24 06:00:39 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:00:49 Waiting for dst bucket item count to get to: 0 curr count: 4642
2021/12/24 06:00:54 Waiting for dst bucket item count to get to: 0 curr count: 2142
2021/12/24 06:00:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:00:55 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:01:05 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 06:01:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLNestedForLoop (128.43s)
=== RUN   TestN1QLAutoClose
2021/12/24 06:01:10 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/24 06:01:10 App: TestN1QLAutoClose got un-deployed
2021/12/24 06:01:11 No eventing-con process running
2021/12/24 06:01:11 Function: TestN1QLAutoClose delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLAutoClose not found"
}
2021/12/24 06:01:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:01:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:01:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:01:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:01:24 Deploying app: TestN1QLAutoClose. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:01:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose stored in temp store"
}
2021/12/24 06:01:30 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/24 06:01:35 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/24 06:01:40 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/24 06:01:45 Waiting for app: TestN1QLAutoClose to get deployed
2021/12/24 06:01:45 App: TestN1QLAutoClose got deployed
2021/12/24 06:01:45 Starting bucket ops to source bucket
2021/12/24 06:01:51 Waiting for dst bucket item count to get to: 500 curr count: 0
2021/12/24 06:01:56 Waiting for dst bucket item count to get to: 500 curr count: 4
2021/12/24 06:02:01 Waiting for dst bucket item count to get to: 500 curr count: 4
2021/12/24 06:02:06 Waiting for dst bucket item count to get to: 500 curr count: 28
2021/12/24 06:02:11 Waiting for dst bucket item count to get to: 500 curr count: 28
2021/12/24 06:02:16 Waiting for dst bucket item count to get to: 500 curr count: 48
2021/12/24 06:02:21 Waiting for dst bucket item count to get to: 500 curr count: 48
2021/12/24 06:02:26 Waiting for dst bucket item count to get to: 500 curr count: 71
2021/12/24 06:02:31 Waiting for dst bucket item count to get to: 500 curr count: 71
2021/12/24 06:02:36 Waiting for dst bucket item count to get to: 500 curr count: 94
2021/12/24 06:02:41 Waiting for dst bucket item count to get to: 500 curr count: 94
2021/12/24 06:02:46 Waiting for dst bucket item count to get to: 500 curr count: 115
2021/12/24 06:02:51 Waiting for dst bucket item count to get to: 500 curr count: 115
2021/12/24 06:02:56 Waiting for dst bucket item count to get to: 500 curr count: 115
2021/12/24 06:03:01 Waiting for dst bucket item count to get to: 500 curr count: 136
2021/12/24 06:03:06 Waiting for dst bucket item count to get to: 500 curr count: 136
2021/12/24 06:03:11 Waiting for dst bucket item count to get to: 500 curr count: 158
2021/12/24 06:03:16 Waiting for dst bucket item count to get to: 500 curr count: 158
2021/12/24 06:03:21 Waiting for dst bucket item count to get to: 500 curr count: 180
2021/12/24 06:03:26 Waiting for dst bucket item count to get to: 500 curr count: 180
2021/12/24 06:03:31 Waiting for dst bucket item count to get to: 500 curr count: 202
2021/12/24 06:03:36 Waiting for dst bucket item count to get to: 500 curr count: 202
2021/12/24 06:03:41 Waiting for dst bucket item count to get to: 500 curr count: 223
2021/12/24 06:03:46 Waiting for dst bucket item count to get to: 500 curr count: 223
2021/12/24 06:03:51 Waiting for dst bucket item count to get to: 500 curr count: 244
2021/12/24 06:03:56 Waiting for dst bucket item count to get to: 500 curr count: 244
2021/12/24 06:04:01 Waiting for dst bucket item count to get to: 500 curr count: 263
2021/12/24 06:04:06 Waiting for dst bucket item count to get to: 500 curr count: 274
2021/12/24 06:04:11 Waiting for dst bucket item count to get to: 500 curr count: 285
2021/12/24 06:04:16 Waiting for dst bucket item count to get to: 500 curr count: 299
2021/12/24 06:04:21 Waiting for dst bucket item count to get to: 500 curr count: 308
2021/12/24 06:04:26 Waiting for dst bucket item count to get to: 500 curr count: 319
2021/12/24 06:04:31 Waiting for dst bucket item count to get to: 500 curr count: 328
2021/12/24 06:04:36 Waiting for dst bucket item count to get to: 500 curr count: 339
2021/12/24 06:04:41 Waiting for dst bucket item count to get to: 500 curr count: 348
2021/12/24 06:04:46 Waiting for dst bucket item count to get to: 500 curr count: 359
2021/12/24 06:04:51 Waiting for dst bucket item count to get to: 500 curr count: 368
2021/12/24 06:04:56 Waiting for dst bucket item count to get to: 500 curr count: 378
2021/12/24 06:05:01 Waiting for dst bucket item count to get to: 500 curr count: 390
2021/12/24 06:05:06 Waiting for dst bucket item count to get to: 500 curr count: 401
2021/12/24 06:05:11 Waiting for dst bucket item count to get to: 500 curr count: 411
2021/12/24 06:05:16 Waiting for dst bucket item count to get to: 500 curr count: 424
2021/12/24 06:05:21 Waiting for dst bucket item count to get to: 500 curr count: 432
2021/12/24 06:05:26 Waiting for dst bucket item count to get to: 500 curr count: 444
2021/12/24 06:05:31 Waiting for dst bucket item count to get to: 500 curr count: 454
2021/12/24 06:05:36 Waiting for dst bucket item count to get to: 500 curr count: 466
2021/12/24 06:05:41 Waiting for dst bucket item count to get to: 500 curr count: 477
2021/12/24 06:05:46 Waiting for dst bucket item count to get to: 500 curr count: 497
2021/12/24 06:05:51 Waiting for dst bucket item count to get to: 500 curr count: 497
2021/12/24 06:05:51 src & dst bucket item count matched up. src bucket count: 500 dst bucket count: 500
2021/12/24 06:05:51 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:05:56 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/24 06:06:02 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/24 06:06:07 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/24 06:06:12 Waiting for app: TestN1QLAutoClose to get un-deployed
2021/12/24 06:06:12 App: TestN1QLAutoClose got un-deployed
2021/12/24 06:06:12 No eventing-con process running
2021/12/24 06:06:12 Function: TestN1QLAutoClose delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLAutoClose deleting in background"
}
2021/12/24 06:06:13 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:06:23 Waiting for dst bucket item count to get to: 0 curr count: 500
2021/12/24 06:06:28 Waiting for dst bucket item count to get to: 0 curr count: 250
2021/12/24 06:06:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:06:30 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:06:40 Waiting for dst bucket item count to get to: 0 curr count: 359
2021/12/24 06:06:45 Waiting for dst bucket item count to get to: 0 curr count: 359
2021/12/24 06:06:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLAutoClose (339.72s)
=== RUN   TestN1QLPosParams
2021/12/24 06:06:50 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/24 06:06:50 App: TestN1QLPosParams got un-deployed
2021/12/24 06:06:50 No eventing-con process running
2021/12/24 06:06:50 Function: TestN1QLPosParams delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLPosParams not found"
}
2021/12/24 06:06:52 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:06:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:06:59 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:07:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:07:04 Deploying app: TestN1QLPosParams. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:07:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams stored in temp store"
}
2021/12/24 06:07:10 Waiting for app: TestN1QLPosParams to get deployed
2021/12/24 06:07:15 Waiting for app: TestN1QLPosParams to get deployed
2021/12/24 06:07:20 Waiting for app: TestN1QLPosParams to get deployed
2021/12/24 06:07:25 Waiting for app: TestN1QLPosParams to get deployed
2021/12/24 06:07:25 App: TestN1QLPosParams got deployed
2021/12/24 06:07:25 Starting bucket ops to source bucket
2021/12/24 06:07:35 Waiting for dst bucket item count to get to: 5000 curr count: 339
2021/12/24 06:07:40 Waiting for dst bucket item count to get to: 5000 curr count: 1029
2021/12/24 06:07:45 Waiting for dst bucket item count to get to: 5000 curr count: 3190
2021/12/24 06:07:45 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 06:07:45 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:07:50 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/24 06:07:55 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/24 06:08:00 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/24 06:08:05 Waiting for app: TestN1QLPosParams to get un-deployed
2021/12/24 06:08:05 App: TestN1QLPosParams got un-deployed
2021/12/24 06:08:05 No eventing-con process running
2021/12/24 06:08:05 Function: TestN1QLPosParams delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLPosParams deleting in background"
}
2021/12/24 06:08:07 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:08:17 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 06:08:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:08:18 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:08:28 Waiting for dst bucket item count to get to: 0 curr count: 4290
2021/12/24 06:08:33 Waiting for dst bucket item count to get to: 0 curr count: 1790
2021/12/24 06:08:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLPosParams (108.29s)
=== RUN   TestN1QLExhaustConnPool
2021/12/24 06:08:38 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/24 06:08:39 App: TestN1QLExhaustConnPool got un-deployed
2021/12/24 06:08:39 No eventing-con process running
2021/12/24 06:08:39 Function: TestN1QLExhaustConnPool delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLExhaustConnPool not found"
}
2021/12/24 06:08:40 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:08:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:08:47 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:08:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:08:52 Deploying app: TestN1QLExhaustConnPool. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:08:53 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool stored in temp store"
}
2021/12/24 06:08:58 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/24 06:09:03 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/24 06:09:08 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/24 06:09:13 Waiting for app: TestN1QLExhaustConnPool to get deployed
2021/12/24 06:09:13 App: TestN1QLExhaustConnPool got deployed
2021/12/24 06:09:13 Starting bucket ops to source bucket
2021/12/24 06:09:19 Waiting for dst bucket item count to get to: 100 curr count: 0
2021/12/24 06:09:24 Waiting for dst bucket item count to get to: 100 curr count: 50
2021/12/24 06:09:24 src & dst bucket item count matched up. src bucket count: 100 dst bucket count: 100
2021/12/24 06:09:24 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:09:29 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/24 06:09:34 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/24 06:09:39 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/24 06:09:44 Waiting for app: TestN1QLExhaustConnPool to get un-deployed
2021/12/24 06:09:44 App: TestN1QLExhaustConnPool got un-deployed
2021/12/24 06:09:44 No eventing-con process running
2021/12/24 06:09:44 Function: TestN1QLExhaustConnPool delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLExhaustConnPool deleting in background"
}
2021/12/24 06:09:46 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:09:56 Waiting for dst bucket item count to get to: 0 curr count: 100
2021/12/24 06:09:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:09:57 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:10:07 Waiting for dst bucket item count to get to: 0 curr count: 50
2021/12/24 06:10:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLExhaustConnPool (93.88s)
=== RUN   TestN1QLGraceSufficientTimeOut
2021/12/24 06:10:12 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/24 06:10:12 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/24 06:10:12 No eventing-con process running
2021/12/24 06:10:12 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestN1QLGraceSufficientTimeOut not found"
}
2021/12/24 06:10:14 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:10:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:10:21 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:10:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:10:26 Starting bucket ops to source bucket
2021/12/24 06:10:43 Starting bucket ops to source bucket
2021/12/24 06:11:00 Deploying app: TestN1QLGraceSufficientTimeOut. Item count src bucket: 30000 dst bucket: 8663
2021/12/24 06:11:01 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut stored in temp store"
}
2021/12/24 06:11:06 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/24 06:11:11 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/24 06:11:16 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/24 06:11:21 Waiting for app: TestN1QLGraceSufficientTimeOut to get deployed
2021/12/24 06:11:21 App: TestN1QLGraceSufficientTimeOut got deployed
2021/12/24 06:11:21 Starting bucket ops to source bucket
2021/12/24 06:11:27 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:32 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:37 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:42 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:47 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:52 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:11:57 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:12:02 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:12:07 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:12:12 Waiting for dst bucket item count to get to: 30001 curr count: 30000
2021/12/24 06:12:12 src & dst bucket item count matched up. src bucket count: 30001 dst bucket count: 30001
2021/12/24 06:12:12 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:12:17 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/24 06:12:22 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/24 06:12:27 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/24 06:12:32 Waiting for app: TestN1QLGraceSufficientTimeOut to get un-deployed
2021/12/24 06:12:32 App: TestN1QLGraceSufficientTimeOut got un-deployed
2021/12/24 06:12:32 No eventing-con process running
2021/12/24 06:12:32 Function: TestN1QLGraceSufficientTimeOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestN1QLGraceSufficientTimeOut deleting in background"
}
2021/12/24 06:12:34 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:12:44 Waiting for dst bucket item count to get to: 0 curr count: 30000
2021/12/24 06:12:49 Waiting for dst bucket item count to get to: 0 curr count: 15000
2021/12/24 06:12:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:12:50 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:13:00 Waiting for dst bucket item count to get to: 0 curr count: 24542
2021/12/24 06:13:05 Waiting for dst bucket item count to get to: 0 curr count: 24542
2021/12/24 06:13:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLGraceSufficientTimeOut (178.11s)
=== RUN   TestN1QLQueryConsistency
2021/12/24 06:13:10 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/24 06:13:10 App: n1qlquery_consistency got un-deployed
2021/12/24 06:13:11 No eventing-con process running
2021/12/24 06:13:11 Function: n1qlquery_consistency delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: n1qlquery_consistency not found"
}
2021/12/24 06:13:12 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:13:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:13:19 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:13:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:13:24 Starting bucket ops to source bucket
2021/12/24 06:13:27 Deploying app: n1qlquery_consistency. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:13:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency stored in temp store"
}
2021/12/24 06:13:33 Waiting for app: n1qlquery_consistency to get deployed
2021/12/24 06:13:38 Waiting for app: n1qlquery_consistency to get deployed
2021/12/24 06:13:43 Waiting for app: n1qlquery_consistency to get deployed
2021/12/24 06:13:48 Waiting for app: n1qlquery_consistency to get deployed
2021/12/24 06:13:48 App: n1qlquery_consistency got deployed
2021/12/24 06:13:53 Waiting for dst bucket item count to get to: 5000 curr count: 0
2021/12/24 06:13:58 Waiting for dst bucket item count to get to: 5000 curr count: 4551
2021/12/24 06:14:03 Waiting for dst bucket item count to get to: 5000 curr count: 4551
2021/12/24 06:14:03 src & dst bucket item count matched up. src bucket count: 5000 dst bucket count: 5000
2021/12/24 06:14:03 http call resp: Get "http://127.0.0.1:9301/api/v1/stats": dial tcp 127.0.0.1:9301: connect: connection refused
2021/12/24 06:14:08 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/24 06:14:14 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/24 06:14:19 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/24 06:14:24 Waiting for app: n1qlquery_consistency to get un-deployed
2021/12/24 06:14:24 App: n1qlquery_consistency got un-deployed
2021/12/24 06:14:24 No eventing-con process running
2021/12/24 06:14:24 Function: n1qlquery_consistency delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: n1qlquery_consistency deleting in background"
}
2021/12/24 06:14:26 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:14:36 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 06:14:36 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:14:37 Context: Bucket flush, response code: 200 dump: 
2021/12/24 06:14:47 Waiting for dst bucket item count to get to: 0 curr count: 5000
2021/12/24 06:14:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestN1QLQueryConsistency (101.80s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	1548.000s

N1ql tests took 0 hours, 25 minutes and 51 seconds.

Starting server
Starting server: attempt 1

Rebalance tests

2021/12/24 06:16:22 Started rebalance
Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:59776->127.0.0.1:9000: read: connection reset by peer
Quota setup Post "http://127.0.0.1:9000/pools/default": read tcp 127.0.0.1:59776->127.0.0.1:9000: read: connection reset by peer
=== RUN   TestDuplicateWithEventingRebIn
2021/12/24 06:16:41 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/24 06:16:41 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/24 06:16:41 No eventing-con process running
2021/12/24 06:16:41 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebIn not found"
}
2021/12/24 06:16:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:16:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:16:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:16:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:16:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:16:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:16:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:16:59 Deploying app: TestDuplicateWithEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:17:00 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn stored in temp store"
}
2021/12/24 06:17:05 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/24 06:17:10 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/24 06:17:15 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/24 06:17:20 Waiting for app: TestDuplicateWithEventingRebIn to get deployed
2021/12/24 06:17:20 App: TestDuplicateWithEventingRebIn got deployed
2021/12/24 06:17:20 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:17:20 Starting bucket ops to source bucket
2021/12/24 06:17:35 Removing node(s): [] from the cluster
2021/12/24 06:17:40 Rebalance progress: 0
2021/12/24 06:17:45 Rebalance progress: 0
2021/12/24 06:17:50 Rebalance progress: 25
2021/12/24 06:17:55 Rebalance progress: 25
2021/12/24 06:18:00 Rebalance progress: 25
2021/12/24 06:18:05 Rebalance progress: 25
2021/12/24 06:18:10 Rebalance progress: 25
2021/12/24 06:18:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:18:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:18:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:18:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/24 06:18:15 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:18:15 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:18:20 Rebalance progress: 25
2021/12/24 06:18:25 Rebalance progress: 25
2021/12/24 06:18:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:18:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:18:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:18:30 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/24 06:18:30 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:18:30 Waiting for src bucket item count to get to: 200000 curr count: 95647
2021/12/24 06:18:35 Waiting for src bucket item count to get to: 200000 curr count: 112697
2021/12/24 06:18:40 Waiting for src bucket item count to get to: 200000 curr count: 112697
2021/12/24 06:18:45 Waiting for src bucket item count to get to: 200000 curr count: 129427
2021/12/24 06:18:50 Waiting for src bucket item count to get to: 200000 curr count: 129427
2021/12/24 06:18:55 Waiting for src bucket item count to get to: 200000 curr count: 146431
2021/12/24 06:19:00 Waiting for src bucket item count to get to: 200000 curr count: 146431
2021/12/24 06:19:05 Waiting for src bucket item count to get to: 200000 curr count: 163344
2021/12/24 06:19:10 Waiting for src bucket item count to get to: 200000 curr count: 163344
2021/12/24 06:19:15 Waiting for src bucket item count to get to: 200000 curr count: 180008
2021/12/24 06:19:20 Waiting for src bucket item count to get to: 200000 curr count: 180008
2021/12/24 06:19:25 Waiting for src bucket item count to get to: 200000 curr count: 196815
2021/12/24 06:19:30 Waiting for src bucket item count to get to: 200000 curr count: 196815
2021/12/24 06:19:40 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:19:45 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:19:50 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:19:55 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:00 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:05 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:10 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:15 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:20 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:25 Waiting for dst bucket item count to get to: 200000 curr count: 200001
2021/12/24 06:20:25 Post SET ops. Expected item count: 200000 got 200001
2021/12/24 06:20:30 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/24 06:20:35 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/24 06:20:40 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/24 06:20:45 Waiting for app: TestDuplicateWithEventingRebIn to get un-deployed
2021/12/24 06:20:45 App: TestDuplicateWithEventingRebIn got un-deployed
2021/12/24 06:20:46 No eventing-con process running
2021/12/24 06:20:46 Function: TestDuplicateWithEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebIn deleting in background"
}
2021/12/24 06:20:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:20:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:20:57 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:20:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:20:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:21:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:21:09 Waiting for dst bucket item count to get to: 0 curr count: 200001
2021/12/24 06:21:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebIn (278.05s)
=== RUN   TestDuplicateWithEventingRebOut
2021/12/24 06:21:14 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:21:15 Removing node(s): [] from the cluster
2021/12/24 06:21:20 Rebalance progress: 100
2021/12/24 06:21:25 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/24 06:21:25 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/24 06:21:25 No eventing-con process running
2021/12/24 06:21:25 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingRebOut not found"
}
2021/12/24 06:21:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:21:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:21:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:21:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:21:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:21:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:21:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:21:43 Deploying app: TestDuplicateWithEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:21:44 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut stored in temp store"
}
2021/12/24 06:21:49 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/24 06:21:54 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/24 06:21:59 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/24 06:22:04 Waiting for app: TestDuplicateWithEventingRebOut to get deployed
2021/12/24 06:22:04 App: TestDuplicateWithEventingRebOut got deployed
2021/12/24 06:22:04 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:22:04 Starting bucket ops to source bucket
2021/12/24 06:22:09 Rebalance progress: 25
2021/12/24 06:22:14 Rebalance progress: 25
2021/12/24 06:22:19 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:22:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:22:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:22:19 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/24 06:22:19 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:22:19 Waiting for src bucket item count to get to: 200000 curr count: 7615
2021/12/24 06:22:24 Waiting for src bucket item count to get to: 200000 curr count: 24031
2021/12/24 06:22:29 Waiting for src bucket item count to get to: 200000 curr count: 24031
2021/12/24 06:22:34 Waiting for src bucket item count to get to: 200000 curr count: 40511
2021/12/24 06:22:39 Waiting for src bucket item count to get to: 200000 curr count: 40511
2021/12/24 06:22:44 Waiting for src bucket item count to get to: 200000 curr count: 57137
2021/12/24 06:22:49 Waiting for src bucket item count to get to: 200000 curr count: 57137
2021/12/24 06:22:54 Waiting for src bucket item count to get to: 200000 curr count: 57137
2021/12/24 06:22:59 Waiting for src bucket item count to get to: 200000 curr count: 73592
2021/12/24 06:23:04 Waiting for src bucket item count to get to: 200000 curr count: 89640
2021/12/24 06:23:09 Waiting for src bucket item count to get to: 200000 curr count: 89640
2021/12/24 06:23:14 Waiting for src bucket item count to get to: 200000 curr count: 106545
2021/12/24 06:23:19 Waiting for src bucket item count to get to: 200000 curr count: 106545
2021/12/24 06:23:24 Waiting for src bucket item count to get to: 200000 curr count: 123111
2021/12/24 06:23:29 Waiting for src bucket item count to get to: 200000 curr count: 123111
2021/12/24 06:23:34 Waiting for src bucket item count to get to: 200000 curr count: 140015
2021/12/24 06:23:39 Waiting for src bucket item count to get to: 200000 curr count: 140015
2021/12/24 06:23:44 Waiting for src bucket item count to get to: 200000 curr count: 156835
2021/12/24 06:23:50 Waiting for src bucket item count to get to: 200000 curr count: 156835
2021/12/24 06:23:55 Waiting for src bucket item count to get to: 200000 curr count: 173325
2021/12/24 06:24:00 Waiting for src bucket item count to get to: 200000 curr count: 173325
2021/12/24 06:24:05 Waiting for src bucket item count to get to: 200000 curr count: 189325
2021/12/24 06:24:10 Waiting for src bucket item count to get to: 200000 curr count: 189325
2021/12/24 06:24:15 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 06:24:15 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 06:24:20 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/24 06:24:25 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/24 06:24:30 Waiting for app: TestDuplicateWithEventingRebOut to get un-deployed
2021/12/24 06:24:30 App: TestDuplicateWithEventingRebOut got un-deployed
2021/12/24 06:24:30 No eventing-con process running
2021/12/24 06:24:30 Function: TestDuplicateWithEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingRebOut deleting in background"
}
2021/12/24 06:24:31 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:24:36 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:24:41 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:24:46 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:24:46 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:24:48 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:24:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:24:58 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:24:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingRebOut (229.22s)
=== RUN   TestDuplicateWithEventingSwapReb
2021/12/24 06:25:08 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/24 06:25:08 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/24 06:25:08 No eventing-con process running
2021/12/24 06:25:08 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithEventingSwapReb not found"
}
2021/12/24 06:25:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:25:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:25:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:25:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:25:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:25:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:25:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:25:27 Deploying app: TestDuplicateWithEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:25:28 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb stored in temp store"
}
2021/12/24 06:25:33 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/24 06:25:38 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/24 06:25:43 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/24 06:25:48 Waiting for app: TestDuplicateWithEventingSwapReb to get deployed
2021/12/24 06:25:48 App: TestDuplicateWithEventingSwapReb got deployed
2021/12/24 06:25:48 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:25:48 Starting bucket ops to source bucket
2021/12/24 06:25:48 Removing node(s): [] from the cluster
2021/12/24 06:25:53 Rebalance progress: 25
2021/12/24 06:25:58 Rebalance progress: 25
2021/12/24 06:26:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:26:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9302/api/v1/stats": dial tcp 127.0.0.1:9302: connect: connection refused
2021/12/24 06:26:03 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:26:03 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:26:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:26:15 Rebalance progress: 16.66666666666667
2021/12/24 06:26:20 Rebalance progress: 16.66666666666667
2021/12/24 06:26:25 Rebalance progress: 16.66666666666667
2021/12/24 06:26:30 Rebalance progress: 64.49275362318842
2021/12/24 06:26:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:26:35 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:26:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:26:40 Rebalance progress: 16.66666666666667
2021/12/24 06:26:45 Rebalance progress: 16.66666666666667
2021/12/24 06:26:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:26:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:26:50 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:26:50 Waiting for src bucket item count to get to: 200000 curr count: 76564
2021/12/24 06:26:55 Waiting for src bucket item count to get to: 200000 curr count: 93234
2021/12/24 06:27:00 Waiting for src bucket item count to get to: 200000 curr count: 93234
2021/12/24 06:27:05 Waiting for src bucket item count to get to: 200000 curr count: 109220
2021/12/24 06:27:10 Waiting for src bucket item count to get to: 200000 curr count: 109220
2021/12/24 06:27:15 Waiting for src bucket item count to get to: 200000 curr count: 125587
2021/12/24 06:27:20 Waiting for src bucket item count to get to: 200000 curr count: 125587
2021/12/24 06:27:25 Waiting for src bucket item count to get to: 200000 curr count: 141811
2021/12/24 06:27:30 Waiting for src bucket item count to get to: 200000 curr count: 141811
2021/12/24 06:27:35 Waiting for src bucket item count to get to: 200000 curr count: 157875
2021/12/24 06:27:40 Waiting for src bucket item count to get to: 200000 curr count: 157875
2021/12/24 06:27:45 Waiting for src bucket item count to get to: 200000 curr count: 173547
2021/12/24 06:27:50 Waiting for src bucket item count to get to: 200000 curr count: 173547
2021/12/24 06:27:55 Waiting for src bucket item count to get to: 200000 curr count: 190174
2021/12/24 06:28:00 Waiting for src bucket item count to get to: 200000 curr count: 190174
2021/12/24 06:28:05 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 06:28:05 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 06:28:10 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/24 06:28:16 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/24 06:28:21 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/24 06:28:26 Waiting for app: TestDuplicateWithEventingSwapReb to get un-deployed
2021/12/24 06:28:26 App: TestDuplicateWithEventingSwapReb got un-deployed
2021/12/24 06:28:26 No eventing-con process running
2021/12/24 06:28:26 Function: TestDuplicateWithEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithEventingSwapReb deleting in background"
}
2021/12/24 06:28:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:28:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:28:37 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:28:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:28:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:28:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:28:49 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:28:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithEventingSwapReb (230.89s)
=== RUN   TestDuplicateWithKVRebIn
2021/12/24 06:28:59 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/24 06:28:59 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/24 06:28:59 No eventing-con process running
2021/12/24 06:28:59 Function: TestDuplicateWithKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebIn not found"
}
2021/12/24 06:29:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:29:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:29:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:29:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:29:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:29:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:29:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:29:18 Deploying app: TestDuplicateWithKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:29:19 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn stored in temp store"
}
2021/12/24 06:29:24 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/24 06:29:29 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/24 06:29:34 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/24 06:29:39 Waiting for app: TestDuplicateWithKVRebIn to get deployed
2021/12/24 06:29:39 App: TestDuplicateWithKVRebIn got deployed
2021/12/24 06:29:39 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 06:29:39 Starting bucket ops to source bucket
2021/12/24 06:29:39 Removing node(s): [] from the cluster
2021/12/24 06:29:44 Rebalance progress: 16.66666666666667
2021/12/24 06:29:49 Rebalance progress: 16.66666666666667
2021/12/24 06:29:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:29:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:29:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:29:54 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:29:54 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:29:59 Rebalance progress: 16.66666666666667
2021/12/24 06:30:04 Rebalance progress: 16.66666666666667
2021/12/24 06:30:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:30:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:30:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:30:09 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:30:09 Waiting for src bucket item count to get to: 200000 curr count: 32229
2021/12/24 06:30:14 Waiting for src bucket item count to get to: 200000 curr count: 48341
2021/12/24 06:30:19 Waiting for src bucket item count to get to: 200000 curr count: 48341
2021/12/24 06:30:24 Waiting for src bucket item count to get to: 200000 curr count: 64763
2021/12/24 06:30:29 Waiting for src bucket item count to get to: 200000 curr count: 64763
2021/12/24 06:30:34 Waiting for src bucket item count to get to: 200000 curr count: 80375
2021/12/24 06:30:39 Waiting for src bucket item count to get to: 200000 curr count: 80375
2021/12/24 06:30:44 Waiting for src bucket item count to get to: 200000 curr count: 80375
2021/12/24 06:30:49 Waiting for src bucket item count to get to: 200000 curr count: 96565
2021/12/24 06:30:54 Waiting for src bucket item count to get to: 200000 curr count: 96565
2021/12/24 06:30:59 Waiting for src bucket item count to get to: 200000 curr count: 113074
2021/12/24 06:31:04 Waiting for src bucket item count to get to: 200000 curr count: 113074
2021/12/24 06:31:09 Waiting for src bucket item count to get to: 200000 curr count: 129867
2021/12/24 06:31:14 Waiting for src bucket item count to get to: 200000 curr count: 146609
2021/12/24 06:31:19 Waiting for src bucket item count to get to: 200000 curr count: 146609
2021/12/24 06:31:24 Waiting for src bucket item count to get to: 200000 curr count: 162964
2021/12/24 06:31:29 Waiting for src bucket item count to get to: 200000 curr count: 162964
2021/12/24 06:31:34 Waiting for src bucket item count to get to: 200000 curr count: 178552
2021/12/24 06:31:39 Waiting for src bucket item count to get to: 200000 curr count: 178552
2021/12/24 06:31:44 Waiting for src bucket item count to get to: 200000 curr count: 194482
2021/12/24 06:31:49 Waiting for src bucket item count to get to: 200000 curr count: 194482
2021/12/24 06:31:54 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 06:31:54 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 06:31:59 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/24 06:32:04 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/24 06:32:09 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/24 06:32:14 Waiting for app: TestDuplicateWithKVRebIn to get un-deployed
2021/12/24 06:32:15 App: TestDuplicateWithKVRebIn got un-deployed
2021/12/24 06:32:15 No eventing-con process running
2021/12/24 06:32:15 Function: TestDuplicateWithKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebIn deleting in background"
}
2021/12/24 06:32:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:32:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:32:26 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:32:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:32:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:32:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:32:38 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:32:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebIn (228.96s)
=== RUN   TestDuplicateWithKVRebOut
2021/12/24 06:32:43 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 06:32:43 Removing node(s): [] from the cluster
2021/12/24 06:32:48 Rebalance progress: 100
2021/12/24 06:32:53 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/24 06:32:53 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/24 06:32:53 No eventing-con process running
2021/12/24 06:32:53 Function: TestDuplicateWithKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVRebOut not found"
}
2021/12/24 06:32:55 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:33:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:33:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:33:02 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:33:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:33:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:33:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:33:12 Deploying app: TestDuplicateWithKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:33:13 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut stored in temp store"
}
2021/12/24 06:33:18 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/24 06:33:23 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/24 06:33:28 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/24 06:33:33 Waiting for app: TestDuplicateWithKVRebOut to get deployed
2021/12/24 06:33:33 App: TestDuplicateWithKVRebOut got deployed
2021/12/24 06:33:33 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:33:33 Starting bucket ops to source bucket
2021/12/24 06:33:38 Rebalance progress: 16.66666666666667
2021/12/24 06:33:43 Rebalance progress: 16.66666666666667
2021/12/24 06:33:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:33:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:33:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:33:48 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:33:48 Waiting for src bucket item count to get to: 200000 curr count: 9497
2021/12/24 06:33:53 Waiting for src bucket item count to get to: 200000 curr count: 25865
2021/12/24 06:33:58 Waiting for src bucket item count to get to: 200000 curr count: 25865
2021/12/24 06:34:03 Waiting for src bucket item count to get to: 200000 curr count: 42065
2021/12/24 06:34:08 Waiting for src bucket item count to get to: 200000 curr count: 42065
2021/12/24 06:34:13 Waiting for src bucket item count to get to: 200000 curr count: 58876
2021/12/24 06:34:18 Waiting for src bucket item count to get to: 200000 curr count: 58876
2021/12/24 06:34:23 Waiting for src bucket item count to get to: 200000 curr count: 58876
2021/12/24 06:34:28 Waiting for src bucket item count to get to: 200000 curr count: 75053
2021/12/24 06:34:33 Waiting for src bucket item count to get to: 200000 curr count: 75053
2021/12/24 06:34:38 Waiting for src bucket item count to get to: 200000 curr count: 91177
2021/12/24 06:34:43 Waiting for src bucket item count to get to: 200000 curr count: 91177
2021/12/24 06:34:48 Waiting for src bucket item count to get to: 200000 curr count: 107821
2021/12/24 06:34:53 Waiting for src bucket item count to get to: 200000 curr count: 107821
2021/12/24 06:34:58 Waiting for src bucket item count to get to: 200000 curr count: 124499
2021/12/24 06:35:03 Waiting for src bucket item count to get to: 200000 curr count: 124499
2021/12/24 06:35:08 Waiting for src bucket item count to get to: 200000 curr count: 141211
2021/12/24 06:35:13 Waiting for src bucket item count to get to: 200000 curr count: 141211
2021/12/24 06:35:18 Waiting for src bucket item count to get to: 200000 curr count: 157903
2021/12/24 06:35:23 Waiting for src bucket item count to get to: 200000 curr count: 157903
2021/12/24 06:35:28 Waiting for src bucket item count to get to: 200000 curr count: 174185
2021/12/24 06:35:33 Waiting for src bucket item count to get to: 200000 curr count: 174185
2021/12/24 06:35:38 Waiting for src bucket item count to get to: 200000 curr count: 190180
2021/12/24 06:35:43 Waiting for src bucket item count to get to: 200000 curr count: 190180
2021/12/24 06:35:48 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 06:35:48 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 06:35:53 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/24 06:35:58 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/24 06:36:03 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/24 06:36:08 Waiting for app: TestDuplicateWithKVRebOut to get un-deployed
2021/12/24 06:36:08 App: TestDuplicateWithKVRebOut got un-deployed
2021/12/24 06:36:09 No eventing-con process running
2021/12/24 06:36:09 Function: TestDuplicateWithKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVRebOut deleting in background"
}
2021/12/24 06:36:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:36:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:36:20 Waiting for dst bucket item count to get to: 0 curr count: 69176
2021/12/24 06:36:25 Waiting for dst bucket item count to get to: 0 curr count: 69176
2021/12/24 06:36:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:36:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:36:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:36:37 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:36:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVRebOut (238.97s)
=== RUN   TestDuplicateWithKVSwapReb
2021/12/24 06:36:47 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/24 06:36:47 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/24 06:36:47 No eventing-con process running
2021/12/24 06:36:47 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateWithKVSwapReb not found"
}
2021/12/24 06:36:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:36:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:36:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:36:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:37:01 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:37:06 Deploying app: TestDuplicateWithKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:37:07 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb stored in temp store"
}
2021/12/24 06:37:12 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/24 06:37:17 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/24 06:37:22 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/24 06:37:27 Waiting for app: TestDuplicateWithKVSwapReb to get deployed
2021/12/24 06:37:27 App: TestDuplicateWithKVSwapReb got deployed
2021/12/24 06:37:27 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 06:37:27 Starting bucket ops to source bucket
2021/12/24 06:37:27 Removing node(s): [] from the cluster
2021/12/24 06:37:32 Rebalance progress: 16.66666666666667
2021/12/24 06:37:37 Rebalance progress: 16.66666666666667
2021/12/24 06:37:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:37:42 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:37:42 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 06:37:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:37:48 Rebalance progress: 16.66666666666667
2021/12/24 06:37:53 Rebalance progress: 16.66666666666667
2021/12/24 06:37:58 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:37:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:37:58 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:37:58 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:38:03 Rebalance progress: 16.66666666666667
2021/12/24 06:38:08 Rebalance progress: 16.66666666666667
2021/12/24 06:38:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:38:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:38:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:38:13 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:38:13 Waiting for src bucket item count to get to: 200000 curr count: 67855
2021/12/24 06:38:18 Waiting for src bucket item count to get to: 200000 curr count: 67855
2021/12/24 06:38:23 Waiting for src bucket item count to get to: 200000 curr count: 83221
2021/12/24 06:38:28 Waiting for src bucket item count to get to: 200000 curr count: 83221
2021/12/24 06:38:33 Waiting for src bucket item count to get to: 200000 curr count: 99398
2021/12/24 06:38:38 Waiting for src bucket item count to get to: 200000 curr count: 99398
2021/12/24 06:38:43 Waiting for src bucket item count to get to: 200000 curr count: 115633
2021/12/24 06:38:48 Waiting for src bucket item count to get to: 200000 curr count: 115633
2021/12/24 06:38:53 Waiting for src bucket item count to get to: 200000 curr count: 132319
2021/12/24 06:38:58 Waiting for src bucket item count to get to: 200000 curr count: 132319
2021/12/24 06:39:03 Waiting for src bucket item count to get to: 200000 curr count: 149008
2021/12/24 06:39:08 Waiting for src bucket item count to get to: 200000 curr count: 149008
2021/12/24 06:39:13 Waiting for src bucket item count to get to: 200000 curr count: 165677
2021/12/24 06:39:18 Waiting for src bucket item count to get to: 200000 curr count: 165677
2021/12/24 06:39:23 Waiting for src bucket item count to get to: 200000 curr count: 181322
2021/12/24 06:39:28 Waiting for src bucket item count to get to: 200000 curr count: 181322
2021/12/24 06:39:33 Waiting for src bucket item count to get to: 200000 curr count: 181322
2021/12/24 06:39:38 Waiting for src bucket item count to get to: 200000 curr count: 197522
2021/12/24 06:39:43 Waiting for src bucket item count to get to: 200000 curr count: 197522
2021/12/24 06:39:48 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 06:39:48 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 06:39:53 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/24 06:39:58 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/24 06:40:03 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/24 06:40:08 Waiting for app: TestDuplicateWithKVSwapReb to get un-deployed
2021/12/24 06:40:08 App: TestDuplicateWithKVSwapReb got un-deployed
2021/12/24 06:40:08 No eventing-con process running
2021/12/24 06:40:08 Function: TestDuplicateWithKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateWithKVSwapReb deleting in background"
}
2021/12/24 06:40:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:40:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:40:20 Waiting for dst bucket item count to get to: 0 curr count: 30120
2021/12/24 06:40:25 Waiting for dst bucket item count to get to: 0 curr count: 30120
2021/12/24 06:40:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:40:27 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:40:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:40:37 Waiting for dst bucket item count to get to: 0 curr count: 200000
2021/12/24 06:40:37 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateWithKVSwapReb (239.75s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpOneByOne
2021/12/24 06:40:47 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 06:40:47 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 06:40:47 No eventing-con process running
2021/12/24 06:40:47 Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/24 06:40:48 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:40:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:40:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:40:55 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:41:00 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:41:05 Deploying app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:41:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/24 06:41:11 Starting bucket ops to source bucket
2021/12/24 06:41:16 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 06:41:21 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 06:41:26 Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 06:41:26 App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:41:26 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:41:26 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:41:27 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:41:32 Rebalance progress: 16.66666666666667
2021/12/24 06:41:37 Rebalance progress: 16.66666666666667
2021/12/24 06:41:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:41:42 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:41:42 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:41:42 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:41:47 Rebalance progress: 16.66666666666667
2021/12/24 06:41:52 Rebalance progress: 16.66666666666667
2021/12/24 06:41:57 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:41:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:41:57 eventingSetup.go:610: http call resp: Get "http://127.0.0.1:9303/api/v1/stats": dial tcp 127.0.0.1:9303: connect: connection refused
2021/12/24 06:41:57 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 06:42:03 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:42:08 Rebalance progress: 12.5
2021/12/24 06:42:13 Rebalance progress: 12.5
2021/12/24 06:42:18 Rebalance progress: 12.5
2021/12/24 06:42:23 Rebalance progress: 99.2827868852459
2021/12/24 06:42:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:42:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:42:33 Rebalance progress: 12.5
2021/12/24 06:42:38 Rebalance progress: 12.5
2021/12/24 06:42:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:42:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:42:48 Rebalance progress: 12.5
2021/12/24 06:42:53 Rebalance progress: 12.5
2021/12/24 06:42:58 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:42:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:42:58 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 06:43:03 Rebalance progress: 12.5
2021/12/24 06:43:08 Rebalance progress: 12.5
2021/12/24 06:43:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:43:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:43:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:43:18 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 06:43:23 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 06:43:28 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 06:43:33 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 06:43:33 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 06:43:33 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:43:33 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/24 06:43:35 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:43:40 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:43:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12160
2021/12/24 06:43:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:43:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:43:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:43:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12160
2021/12/24 06:43:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpOneByOne (200.09s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce
2021/12/24 06:44:07 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/24 06:44:07 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/24 06:44:07 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:44:07 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce not found"
}
2021/12/24 06:44:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:44:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:44:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:44:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:44:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:44:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:44:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:44:25 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:44:27 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce stored in temp store"
}
2021/12/24 06:44:32 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 06:44:37 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/24 06:44:42 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/24 06:44:47 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get deployed
2021/12/24 06:44:47 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:44:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:44:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:44:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:44:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:44:47 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 06:44:48 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:44:53 Rebalance progress: 12.5
2021/12/24 06:44:58 Rebalance progress: 12.5
2021/12/24 06:45:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:45:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:45:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:45:03 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/24 06:45:08 Rebalance progress: 12.5
2021/12/24 06:45:13 Rebalance progress: 12.5
2021/12/24 06:45:18 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:45:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:45:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:45:23 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/24 06:45:28 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/24 06:45:33 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/24 06:45:38 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce to get un-deployed
2021/12/24 06:45:38 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce got un-deployed
2021/12/24 06:45:38 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:45:38 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce deleting in background"
}
2021/12/24 06:45:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:45:45 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:45:50 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2
2021/12/24 06:45:55 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2
2021/12/24 06:45:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:45:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:46:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4618
2021/12/24 06:46:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpAllAtOnce (129.76s)
=== RUN   TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings
2021/12/24 06:46:17 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 06:46:17 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/24 06:46:17 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:46:17 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings not found"
}
2021/12/24 06:46:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:46:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:46:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:46:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:46:35 eventingSetup.go:125: Deploying app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:46:36 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings stored in temp store"
}
2021/12/24 06:46:41 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 06:46:46 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/24 06:46:51 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/24 06:46:56 clusterSetup.go:346: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get deployed
2021/12/24 06:46:56 clusterSetup.go:354: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:46:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:46:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:46:57 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:47:02 Rebalance progress: 12.5
2021/12/24 06:47:07 Rebalance progress: 12.5
2021/12/24 06:47:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:47:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:47:12 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:47:17 Rebalance progress: 12.5
2021/12/24 06:47:22 Rebalance progress: 12.5
2021/12/24 06:47:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:47:27 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 06:47:27 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:47:32 Rebalance progress: 12.5
2021/12/24 06:47:37 Rebalance progress: 12.5
2021/12/24 06:47:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:47:42 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:47:47 Rebalance progress: 12.5
2021/12/24 06:47:52 Rebalance progress: 12.5
2021/12/24 06:47:57 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:47:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:47:57 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:48:02 Rebalance progress: 12.5
2021/12/24 06:48:07 Rebalance progress: 12.5
2021/12/24 06:48:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:48:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:48:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:48:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 06:48:18 Rebalance progress: 12.5
2021/12/24 06:48:23 Rebalance progress: 12.5
2021/12/24 06:48:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:48:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:48:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:48:33 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 06:48:38 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 06:48:43 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 06:48:48 clusterSetup.go:397: Waiting for app: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings to get un-deployed
2021/12/24 06:48:48 clusterSetup.go:405: App: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings got un-deployed
2021/12/24 06:48:48 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:48:48 eventingSetup.go:418: Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings deleting in background"
}
2021/12/24 06:48:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:48:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:48:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:48:56 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:01 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:49:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10619
2021/12/24 06:49:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebKVOpsOnUpdateBucketOpNonDefaultSettings (179.75s)
=== RUN   TestEventingSwapRebOnUpdateBucketOp
2021/12/24 06:49:16 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/24 06:49:16 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/24 06:49:16 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:49:16 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebOnUpdateBucketOp not found"
}
2021/12/24 06:49:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:49:23 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:49:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:49:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:49:30 eventingSetup.go:125: Deploying app: TestEventingSwapRebOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:49:31 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp stored in temp store"
}
2021/12/24 06:49:36 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 06:49:41 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/24 06:49:46 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/24 06:49:51 clusterSetup.go:346: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get deployed
2021/12/24 06:49:51 clusterSetup.go:354: App: TestEventingSwapRebOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:51 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:49:51 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:49:51 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:49:51 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:49:56 Rebalance progress: 12.5
2021/12/24 06:50:01 Rebalance progress: 12.5
2021/12/24 06:50:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:50:06 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:50:07 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:50:12 Rebalance progress: 12.5
2021/12/24 06:50:17 Rebalance progress: 12.5
2021/12/24 06:50:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:50:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:50:27 Rebalance progress: 12.5
2021/12/24 06:50:32 Rebalance progress: 12.5
2021/12/24 06:50:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:50:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:50:42 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/24 06:50:47 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/24 06:50:53 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/24 06:50:58 clusterSetup.go:397: Waiting for app: TestEventingSwapRebOnUpdateBucketOp to get un-deployed
2021/12/24 06:50:58 clusterSetup.go:405: App: TestEventingSwapRebOnUpdateBucketOp got un-deployed
2021/12/24 06:50:58 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:50:58 eventingSetup.go:418: Function: TestEventingSwapRebOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebOnUpdateBucketOp deleting in background"
}
2021/12/24 06:50:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:51:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:51:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6127
2021/12/24 06:51:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:51:11 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:51:16 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:51:21 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6127
2021/12/24 06:51:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6127
2021/12/24 06:51:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebOnUpdateBucketOp (139.71s)
=== RUN   TestMetaRollbackWithEventingReb
2021/12/24 06:51:31 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:51:31 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:51:36 Rebalance progress: 100
2021/12/24 06:51:41 Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/24 06:51:41 App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/24 06:51:42 No eventing-con process running
2021/12/24 06:51:42 Function: TestMetaRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaRollbackWithEventingReb not found"
}
2021/12/24 06:51:43 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:51:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:51:48 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:51:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:51:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:51:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:51:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:51:55 Deploying app: TestMetaRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:51:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb stored in temp store"
}
2021/12/24 06:52:01 Starting bucket ops to source bucket
2021/12/24 06:52:06 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/24 06:52:11 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/24 06:52:16 Waiting for app: TestMetaRollbackWithEventingReb to get deployed
2021/12/24 06:52:16 App: TestMetaRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:52:16 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:52:17 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:52:22 Rebalance progress: 12.5
2021/12/24 06:52:27 Rebalance progress: 12.5
2021/12/24 06:52:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:52:32 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/24 06:52:32 bucketOps.go:134: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/24 06:52:37 Rebalance progress: 12.5
2021/12/24 06:52:42 Rebalance progress: 12.5
2021/12/24 06:52:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:52:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:52:52 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/24 06:52:57 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/24 06:53:02 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/24 06:53:07 clusterSetup.go:397: Waiting for app: TestMetaRollbackWithEventingReb to get un-deployed
2021/12/24 06:53:07 clusterSetup.go:405: App: TestMetaRollbackWithEventingReb got un-deployed
2021/12/24 06:53:07 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:53:07 eventingSetup.go:418: Function: TestMetaRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaRollbackWithEventingReb deleting in background"
}
2021/12/24 06:53:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:53:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:53:19 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4552
2021/12/24 06:53:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:53:20 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:53:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:53:30 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2771
2021/12/24 06:53:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 2771
2021/12/24 06:53:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaRollbackWithEventingReb (129.44s)
=== RUN   TestMetaPartialRollbackWithEventingReb
2021/12/24 06:53:41 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:53:41 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:53:46 Rebalance progress: 100
2021/12/24 06:53:51 Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/24 06:53:51 App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/24 06:53:51 No eventing-con process running
2021/12/24 06:53:51 Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestMetaPartialRollbackWithEventingReb not found"
}
2021/12/24 06:53:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:53:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:53:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:54:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:54:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:54:05 Deploying app: TestMetaPartialRollbackWithEventingReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:54:06 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb stored in temp store"
}
2021/12/24 06:54:11 Starting bucket ops to source bucket
2021/12/24 06:54:16 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/24 06:54:21 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/24 06:54:26 Waiting for app: TestMetaPartialRollbackWithEventingReb to get deployed
2021/12/24 06:54:26 App: TestMetaPartialRollbackWithEventingReb got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:54:26 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:54:26 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:54:31 Rebalance progress: 12.5
2021/12/24 06:54:36 Rebalance progress: 12.5
2021/12/24 06:54:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:54:41 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/24 06:54:41 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/24 06:54:46 Rebalance progress: 12.5
2021/12/24 06:54:51 Rebalance progress: 12.5
2021/12/24 06:54:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:54:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:54:56 bucketOps.go:65: Metakv lookup failed, err: Get "/_metakv/eventing/tempApps/TestMetaPartialRollbackWithEventingReb/0": Unable to initialize cbauth's revrpc: cbauth environment variable CBAUTH_REVRPC_URL is not set
2021/12/24 06:55:01 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/24 06:55:07 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/24 06:55:12 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/24 06:55:17 clusterSetup.go:397: Waiting for app: TestMetaPartialRollbackWithEventingReb to get un-deployed
2021/12/24 06:55:17 clusterSetup.go:405: App: TestMetaPartialRollbackWithEventingReb got un-deployed
2021/12/24 06:55:17 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:55:17 eventingSetup.go:418: Function: TestMetaPartialRollbackWithEventingReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestMetaPartialRollbackWithEventingReb deleting in background"
}
2021/12/24 06:55:18 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:55:23 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:55:28 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 38216
2021/12/24 06:55:28 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:55:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:55:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:55:40 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8129
2021/12/24 06:55:45 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8129
2021/12/24 06:55:45 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestMetaPartialRollbackWithEventingReb (129.66s)
=== RUN   TestEventingRebMultipleHandlersOneByOne
2021/12/24 06:55:55 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/24 06:55:55 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/24 06:55:55 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:55:55 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op not found"
}
2021/12/24 06:55:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:56:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:56:04 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:56:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:56:14 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:56:15 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:56:15 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 06:56:16 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update stored in temp store"
}
2021/12/24 06:56:21 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 06:56:26 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/24 06:56:31 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/24 06:56:36 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get deployed
2021/12/24 06:56:36 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got deployed
2021/12/24 06:56:41 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/24 06:56:46 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/24 06:56:51 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get deployed
2021/12/24 06:56:51 clusterSetup.go:354: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:51 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:56:51 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:56:51 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 06:56:52 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:56:57 Rebalance progress: 12.5
2021/12/24 06:57:02 Rebalance progress: 12.5
2021/12/24 06:57:07 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:57:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 06:57:07 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:57:12 Rebalance progress: 12.5
2021/12/24 06:57:17 Rebalance progress: 12.5
2021/12/24 06:57:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:57:22 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 06:57:22 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 06:57:27 Rebalance progress: 12.5
2021/12/24 06:57:32 Rebalance progress: 12.5
2021/12/24 06:57:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:57:37 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 06:57:43 Rebalance progress: 12.5
2021/12/24 06:57:48 Rebalance progress: 12.5
2021/12/24 06:57:53 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:57:53 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:57:53 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 06:57:58 Rebalance progress: 12.5
2021/12/24 06:58:03 Rebalance progress: 12.5
2021/12/24 06:58:08 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:58:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:58:08 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:58:08 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 06:58:13 Rebalance progress: 12.5
2021/12/24 06:58:18 Rebalance progress: 12.5
2021/12/24 06:58:23 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:58:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:58:23 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 06:58:28 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/24 06:58:33 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/24 06:58:38 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/24 06:58:43 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op to get un-deployed
2021/12/24 06:58:43 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op got un-deployed
2021/12/24 06:58:43 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:58:43 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnesys_test_bucket_op deleting in background"
}
2021/12/24 06:58:46 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:58:51 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:58:56 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12171
2021/12/24 06:58:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:58:58 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:59:03 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:59:08 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12171
2021/12/24 06:59:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:59:13 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/24 06:59:18 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/24 06:59:23 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/24 06:59:28 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update to get un-deployed
2021/12/24 06:59:28 clusterSetup.go:405: App: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update got un-deployed
2021/12/24 06:59:28 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:59:28 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersOneByOnebucket_op_on_update deleting in background"
}
2021/12/24 06:59:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:59:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:59:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 06:59:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:59:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:59:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersOneByOne (236.53s)
=== RUN   TestEventingRebMultipleHandlersAllAtOnce
2021/12/24 06:59:52 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/24 06:59:52 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/24 06:59:52 eventingSetup.go:46: No eventing-con process running
2021/12/24 06:59:52 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op not found"
}
2021/12/24 06:59:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 06:59:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 06:59:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:00:00 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:00:05 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:00:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:00:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:00:10 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:00:11 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:00:11 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:00:11 eventingSetup.go:125: Deploying app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:00:13 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update stored in temp store"
}
2021/12/24 07:00:18 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:00:23 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/24 07:00:28 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/24 07:00:33 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get deployed
2021/12/24 07:00:33 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got deployed
2021/12/24 07:00:38 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/24 07:00:43 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/24 07:00:48 clusterSetup.go:346: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get deployed
2021/12/24 07:00:48 clusterSetup.go:354: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:00:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:00:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:00:48 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:00:48 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:00:49 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: eventing to the cluster
2021/12/24 07:00:49 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:00:54 Rebalance progress: 12.5
2021/12/24 07:00:59 Rebalance progress: 12.5
2021/12/24 07:01:04 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:01:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:01:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:01:04 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/24 07:01:09 Rebalance progress: 12.5
2021/12/24 07:01:14 Rebalance progress: 12.5
2021/12/24 07:01:19 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:01:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:01:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:01:24 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/24 07:01:30 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/24 07:01:35 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/24 07:01:40 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op to get un-deployed
2021/12/24 07:01:40 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op got un-deployed
2021/12/24 07:01:40 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:01:40 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncesys_test_bucket_op deleting in background"
}
2021/12/24 07:01:43 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:01:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:01:53 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6164
2021/12/24 07:01:58 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6164
2021/12/24 07:01:58 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:01:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:02:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:02:04 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:02:10 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/24 07:02:15 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/24 07:02:20 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/24 07:02:25 clusterSetup.go:397: Waiting for app: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update to get un-deployed
2021/12/24 07:02:25 clusterSetup.go:405: App: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update got un-deployed
2021/12/24 07:02:25 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:02:25 eventingSetup.go:418: Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultipleHandlersAllAtOncebucket_op_on_update deleting in background"
}
2021/12/24 07:02:27 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:02:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:02:32 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:02:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:02:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:02:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultipleHandlersAllAtOnce (176.59s)
=== RUN   TestEventingSwapRebMultipleHandlers
2021/12/24 07:02:48 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/24 07:02:48 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/24 07:02:48 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:02:48 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update not found"
}
2021/12/24 07:02:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:02:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:02:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:02:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:03:02 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:03:07 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/24 07:03:07 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/24 07:03:07 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:03:07 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op not found"
}
2021/12/24 07:03:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:03:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:03:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:03:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:03:20 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlersbucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:03:21 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:03:21 eventingSetup.go:125: Deploying app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:03:23 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op stored in temp store"
}
2021/12/24 07:03:28 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:03:33 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/24 07:03:38 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/24 07:03:43 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get deployed
2021/12/24 07:03:43 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got deployed
2021/12/24 07:03:48 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/24 07:03:53 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/24 07:03:58 clusterSetup.go:346: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get deployed
2021/12/24 07:03:58 clusterSetup.go:354: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:58 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:03:58 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:03:58 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:03:58 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:04:03 Rebalance progress: 12.5
2021/12/24 07:04:08 Rebalance progress: 12.5
2021/12/24 07:04:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:04:13 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:04:14 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:04:19 Rebalance progress: 12.5
2021/12/24 07:04:24 Rebalance progress: 12.5
2021/12/24 07:04:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:04:29 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:04:34 Rebalance progress: 12.5
2021/12/24 07:04:39 Rebalance progress: 12.5
2021/12/24 07:04:44 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:04:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:04:49 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/24 07:04:54 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/24 07:04:59 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/24 07:05:04 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlersbucket_op_on_update to get un-deployed
2021/12/24 07:05:04 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlersbucket_op_on_update got un-deployed
2021/12/24 07:05:04 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:05:04 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlersbucket_op_on_update deleting in background"
}
2021/12/24 07:05:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:05:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:05:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7624
2021/12/24 07:05:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:05:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:05:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:05:29 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7624
2021/12/24 07:05:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:05:34 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/24 07:05:39 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/24 07:05:44 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/24 07:05:50 clusterSetup.go:397: Waiting for app: TestEventingSwapRebMultipleHandlerssys_test_bucket_op to get un-deployed
2021/12/24 07:05:50 clusterSetup.go:405: App: TestEventingSwapRebMultipleHandlerssys_test_bucket_op got un-deployed
2021/12/24 07:05:50 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:05:50 eventingSetup.go:418: Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebMultipleHandlerssys_test_bucket_op deleting in background"
}
2021/12/24 07:05:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:05:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:05:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:05:58 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:03 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:06:03 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebMultipleHandlers (204.76s)
=== RUN   TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/24 07:06:13 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:06:13 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 07:06:13 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:06:13 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/24 07:06:15 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:06:20 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:06:22 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:27 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:06:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:06:32 eventingSetup.go:125: Deploying app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:06:33 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/24 07:06:38 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:06:43 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:06:48 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:06:53 clusterSetup.go:346: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:06:53 clusterSetup.go:354: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:06:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:06:53 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:06:53 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:07:13 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:44 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:07:44 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:07:49 Rebalance progress: 12.5
2021/12/24 07:07:54 Rebalance progress: 12.5
2021/12/24 07:07:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:07:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:07:59 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:07:59 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:08:05 Rebalance progress: 12.5
2021/12/24 07:08:10 Rebalance progress: 12.5
2021/12/24 07:08:15 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:08:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:08:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:08:15 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:08:20 Rebalance progress: 12.5
2021/12/24 07:08:25 Rebalance progress: 12.5
2021/12/24 07:08:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:08:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:08:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:08:35 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:08:40 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:08:45 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:08:50 clusterSetup.go:397: Waiting for app: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:08:50 clusterSetup.go:405: App: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 07:08:50 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:08:50 eventingSetup.go:418: Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/24 07:08:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:08:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:09:02 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11183
2021/12/24 07:09:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11183
2021/12/24 07:09:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:09:08 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:09:13 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:09:18 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 11183
2021/12/24 07:09:18 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebStopStartKVOpsOnUpdateBucketOpOneByOne (195.30s)
=== RUN   TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/24 07:09:28 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:09:28 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 07:09:28 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:09:28 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne not found"
}
2021/12/24 07:09:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:09:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:09:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:09:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:09:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:09:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:09:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:09:47 eventingSetup.go:125: Deploying app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:09:48 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne stored in temp store"
}
2021/12/24 07:09:53 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:09:58 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:10:03 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:10:08 clusterSetup.go:346: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get deployed
2021/12/24 07:10:08 clusterSetup.go:354: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:10:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:10:08 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:10:28 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:10:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:10:39 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:10:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:11:00 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:11:10 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:11:10 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:11:30 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:31 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:31 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:11:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:11:41 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:11:41 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:12:01 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:12:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:12:13 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:12:33 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:33 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:12:43 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:12:48 Rebalance progress: 12.5
2021/12/24 07:12:53 Rebalance progress: 12.5
2021/12/24 07:12:58 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:12:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:12:58 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:12:58 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:13:03 Rebalance progress: 12.5
2021/12/24 07:13:08 Rebalance progress: 12.5
2021/12/24 07:13:13 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:13:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:13:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:13:13 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:13:19 Rebalance progress: 12.5
2021/12/24 07:13:24 Rebalance progress: 12.5
2021/12/24 07:13:29 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:13:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:13:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:13:34 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:13:39 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:13:44 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:13:49 clusterSetup.go:397: Waiting for app: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne to get un-deployed
2021/12/24 07:13:49 clusterSetup.go:405: App: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne got un-deployed
2021/12/24 07:13:49 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:13:49 eventingSetup.go:418: Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne deleting in background"
}
2021/12/24 07:13:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:13:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:14:01 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 13685
2021/12/24 07:14:06 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 13685
2021/12/24 07:14:06 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:14:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:14:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:14:17 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 21550
2021/12/24 07:14:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebMultiStopStartKVOpsOnUpdateBucketOpOneByOne (299.05s)
=== RUN   TestEventingFailoverOnUpdateBucketOp
2021/12/24 07:14:27 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/24 07:14:27 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/24 07:14:27 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:14:27 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingFailoverOnUpdateBucketOp not found"
}
2021/12/24 07:14:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:14:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:14:34 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:14:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:14:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:14:41 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:14:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:14:46 eventingSetup.go:125: Deploying app: TestEventingFailoverOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:14:47 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp stored in temp store"
}
2021/12/24 07:14:52 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:14:57 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/24 07:15:02 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/24 07:15:07 clusterSetup.go:346: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get deployed
2021/12/24 07:15:07 clusterSetup.go:354: App: TestEventingFailoverOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:07 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:15:07 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:15:07 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:15:12 Rebalance progress: 12.5
2021/12/24 07:15:17 Rebalance progress: 12.5
2021/12/24 07:15:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:15:22 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:32 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:15:32 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:15:37 Rebalance progress: 12.5
2021/12/24 07:15:42 Rebalance progress: 12.5
2021/12/24 07:15:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:15:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:15:48 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:15:48 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:15:53 Rebalance progress: 12.5
2021/12/24 07:15:58 Rebalance progress: 12.5
2021/12/24 07:16:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:16:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:16:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:16:03 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:16:08 Rebalance progress: 12.5
2021/12/24 07:16:13 Rebalance progress: 12.5
2021/12/24 07:16:18 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:16:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:16:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:16:23 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/24 07:16:28 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/24 07:16:33 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/24 07:16:38 clusterSetup.go:397: Waiting for app: TestEventingFailoverOnUpdateBucketOp to get un-deployed
2021/12/24 07:16:38 clusterSetup.go:405: App: TestEventingFailoverOnUpdateBucketOp got un-deployed
2021/12/24 07:16:38 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:16:38 eventingSetup.go:418: Function: TestEventingFailoverOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingFailoverOnUpdateBucketOp deleting in background"
}
2021/12/24 07:16:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:16:45 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:16:50 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1776
2021/12/24 07:16:55 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1776
2021/12/24 07:16:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:16:57 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:02 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:17:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8609
2021/12/24 07:17:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingFailoverOnUpdateBucketOp (169.47s)
=== RUN   TestEventingKVRebalanceOnUpdateBucketOp
2021/12/24 07:17:17 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/24 07:17:17 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/24 07:17:17 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:17:17 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingKVRebalanceOnUpdateBucketOp not found"
}
2021/12/24 07:17:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:17:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:17:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:17:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:17:35 eventingSetup.go:125: Deploying app: TestEventingKVRebalanceOnUpdateBucketOp. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:17:37 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp stored in temp store"
}
2021/12/24 07:17:42 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:17:47 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/24 07:17:52 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/24 07:17:57 clusterSetup.go:346: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get deployed
2021/12/24 07:17:57 clusterSetup.go:354: App: TestEventingKVRebalanceOnUpdateBucketOp got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:17:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:17:57 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing,kv to the cluster
2021/12/24 07:17:57 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:18:02 Rebalance progress: 12.5
2021/12/24 07:18:07 Rebalance progress: 12.5
2021/12/24 07:18:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:18:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing,kv to the cluster
2021/12/24 07:18:12 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:18:18 Rebalance progress: 12.5
2021/12/24 07:18:23 Rebalance progress: 12.5
2021/12/24 07:18:28 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:18:28 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:18:33 Rebalance progress: 12.5
2021/12/24 07:18:38 Rebalance progress: 12.5
2021/12/24 07:18:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:18:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:18:48 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/24 07:18:53 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/24 07:18:58 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/24 07:19:03 clusterSetup.go:397: Waiting for app: TestEventingKVRebalanceOnUpdateBucketOp to get un-deployed
2021/12/24 07:19:03 clusterSetup.go:405: App: TestEventingKVRebalanceOnUpdateBucketOp got un-deployed
2021/12/24 07:19:03 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:19:03 eventingSetup.go:418: Function: TestEventingKVRebalanceOnUpdateBucketOp delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingKVRebalanceOnUpdateBucketOp deleting in background"
}
2021/12/24 07:19:05 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:19:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:19:15 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6092
2021/12/24 07:19:15 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:19:16 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:19:21 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:19:26 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6092
2021/12/24 07:19:26 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingKVRebalanceOnUpdateBucketOp (139.57s)
=== RUN   TestEventingRebInWithOps
2021/12/24 07:19:36 clusterSetup.go:397: Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/24 07:19:36 clusterSetup.go:405: App: TestEventingRebInWithOps got un-deployed
2021/12/24 07:19:36 eventingSetup.go:46: No eventing-con process running
2021/12/24 07:19:36 eventingSetup.go:418: Function: TestEventingRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithOps not found"
}
2021/12/24 07:19:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:19:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:19:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:19:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:19:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:19:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:19:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:19:55 eventingSetup.go:125: Deploying app: TestEventingRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:19:56 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps stored in temp store"
}
2021/12/24 07:20:01 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/24 07:20:06 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/24 07:20:11 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/24 07:20:16 clusterSetup.go:346: Waiting for app: TestEventingRebInWithOps to get deployed
2021/12/24 07:20:16 clusterSetup.go:354: App: TestEventingRebInWithOps got deployed
2021/12/24 07:20:16 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:20:16 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 07:20:17 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 07:20:22 Rebalance progress: 12.5
2021/12/24 07:20:27 Rebalance progress: 12.5
2021/12/24 07:20:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:20:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:20:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:20:32 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:20:37 Rebalance progress: 12.5
2021/12/24 07:20:42 Rebalance progress: 12.5
2021/12/24 07:20:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:20:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:20:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:20:47 Waiting for src bucket item count to get to: 200000 curr count: 35028
2021/12/24 07:20:52 Waiting for src bucket item count to get to: 200000 curr count: 50896
2021/12/24 07:20:57 Waiting for src bucket item count to get to: 200000 curr count: 50896
2021/12/24 07:21:02 Waiting for src bucket item count to get to: 200000 curr count: 67303
2021/12/24 07:21:07 Waiting for src bucket item count to get to: 200000 curr count: 67303
2021/12/24 07:21:12 Waiting for src bucket item count to get to: 200000 curr count: 82405
2021/12/24 07:21:17 Waiting for src bucket item count to get to: 200000 curr count: 82405
2021/12/24 07:21:22 Waiting for src bucket item count to get to: 200000 curr count: 98998
2021/12/24 07:21:27 Waiting for src bucket item count to get to: 200000 curr count: 98998
2021/12/24 07:21:32 Waiting for src bucket item count to get to: 200000 curr count: 115498
2021/12/24 07:21:37 Waiting for src bucket item count to get to: 200000 curr count: 115498
2021/12/24 07:21:42 Waiting for src bucket item count to get to: 200000 curr count: 131895
2021/12/24 07:21:47 Waiting for src bucket item count to get to: 200000 curr count: 131895
2021/12/24 07:21:52 Waiting for src bucket item count to get to: 200000 curr count: 131895
2021/12/24 07:21:57 Waiting for src bucket item count to get to: 200000 curr count: 148355
2021/12/24 07:22:02 Waiting for src bucket item count to get to: 200000 curr count: 148355
2021/12/24 07:22:07 Waiting for src bucket item count to get to: 200000 curr count: 164467
2021/12/24 07:22:12 Waiting for src bucket item count to get to: 200000 curr count: 164467
2021/12/24 07:22:17 Waiting for src bucket item count to get to: 200000 curr count: 179647
2021/12/24 07:22:22 Waiting for src bucket item count to get to: 200000 curr count: 179647
2021/12/24 07:22:27 Waiting for src bucket item count to get to: 200000 curr count: 195955
2021/12/24 07:22:32 Waiting for src bucket item count to get to: 200000 curr count: 195955
2021/12/24 07:22:37 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 07:22:37 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 07:22:37 Starting bucket ops to source bucket
2021/12/24 07:26:23 Waiting for dst bucket item count to get to: 0 curr count: 15813
2021/12/24 07:26:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:26:23 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 07:26:28 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/24 07:26:33 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/24 07:26:38 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/24 07:26:43 Waiting for app: TestEventingRebInWithOps to get un-deployed
2021/12/24 07:26:43 App: TestEventingRebInWithOps got un-deployed
2021/12/24 07:26:43 No eventing-con process running
2021/12/24 07:26:43 Function: TestEventingRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithOps deleting in background"
}
2021/12/24 07:26:45 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:26:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:26:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:26:51 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:26:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:26:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithOps (450.06s)
=== RUN   TestEventingRebOutWithOps
2021/12/24 07:27:01 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:27:02 Removing node(s): [] from the cluster
2021/12/24 07:27:07 Rebalance progress: 100
2021/12/24 07:27:12 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/24 07:27:12 App: TestEventingRebOutWithOps got un-deployed
2021/12/24 07:27:12 No eventing-con process running
2021/12/24 07:27:12 Function: TestEventingRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithOps not found"
}
2021/12/24 07:27:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:27:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:27:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:27:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:27:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:27:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:27:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:27:30 Deploying app: TestEventingRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:27:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps stored in temp store"
}
2021/12/24 07:27:36 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/24 07:27:42 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/24 07:27:47 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/24 07:27:52 Waiting for app: TestEventingRebOutWithOps to get deployed
2021/12/24 07:27:52 App: TestEventingRebOutWithOps got deployed
2021/12/24 07:27:52 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:27:52 Starting bucket ops to source bucket
2021/12/24 07:27:57 Rebalance progress: 12.5
2021/12/24 07:28:02 Rebalance progress: 12.5
2021/12/24 07:28:07 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:28:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:28:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:28:07 Waiting for src bucket item count to get to: 200000 curr count: 11649
2021/12/24 07:28:12 Waiting for src bucket item count to get to: 200000 curr count: 27867
2021/12/24 07:28:17 Waiting for src bucket item count to get to: 200000 curr count: 27867
2021/12/24 07:28:22 Waiting for src bucket item count to get to: 200000 curr count: 44200
2021/12/24 07:28:27 Waiting for src bucket item count to get to: 200000 curr count: 44200
2021/12/24 07:28:32 Waiting for src bucket item count to get to: 200000 curr count: 60641
2021/12/24 07:28:37 Waiting for src bucket item count to get to: 200000 curr count: 60641
2021/12/24 07:28:42 Waiting for src bucket item count to get to: 200000 curr count: 76338
2021/12/24 07:28:47 Waiting for src bucket item count to get to: 200000 curr count: 76338
2021/12/24 07:28:52 Waiting for src bucket item count to get to: 200000 curr count: 76338
2021/12/24 07:28:57 Waiting for src bucket item count to get to: 200000 curr count: 92538
2021/12/24 07:29:02 Waiting for src bucket item count to get to: 200000 curr count: 92538
2021/12/24 07:29:07 Waiting for src bucket item count to get to: 200000 curr count: 109123
2021/12/24 07:29:12 Waiting for src bucket item count to get to: 200000 curr count: 109123
2021/12/24 07:29:17 Waiting for src bucket item count to get to: 200000 curr count: 125845
2021/12/24 07:29:22 Waiting for src bucket item count to get to: 200000 curr count: 125845
2021/12/24 07:29:27 Waiting for src bucket item count to get to: 200000 curr count: 142118
2021/12/24 07:29:32 Waiting for src bucket item count to get to: 200000 curr count: 142118
2021/12/24 07:29:37 Waiting for src bucket item count to get to: 200000 curr count: 158753
2021/12/24 07:29:42 Waiting for src bucket item count to get to: 200000 curr count: 158753
2021/12/24 07:29:47 Waiting for src bucket item count to get to: 200000 curr count: 174255
2021/12/24 07:29:52 Waiting for src bucket item count to get to: 200000 curr count: 174255
2021/12/24 07:29:57 Waiting for src bucket item count to get to: 200000 curr count: 190359
2021/12/24 07:30:02 Waiting for src bucket item count to get to: 200000 curr count: 190359
2021/12/24 07:30:07 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 07:30:07 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 07:30:07 Starting bucket ops to source bucket
2021/12/24 07:33:52 Waiting for dst bucket item count to get to: 0 curr count: 14593
2021/12/24 07:33:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:33:52 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 07:33:57 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/24 07:34:02 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/24 07:34:08 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/24 07:34:13 Waiting for app: TestEventingRebOutWithOps to get un-deployed
2021/12/24 07:34:13 App: TestEventingRebOutWithOps got un-deployed
2021/12/24 07:34:13 No eventing-con process running
2021/12/24 07:34:13 Function: TestEventingRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithOps deleting in background"
}
2021/12/24 07:34:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:34:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:34:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:34:21 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:34:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:34:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithOps (449.57s)
=== RUN   TestEventingSwapRebWithOps
2021/12/24 07:34:36 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/24 07:34:36 App: TestEventingSwapRebWithOps got un-deployed
2021/12/24 07:34:36 No eventing-con process running
2021/12/24 07:34:36 Function: TestEventingSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithOps not found"
}
2021/12/24 07:34:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:34:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:34:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:34:45 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:34:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:34:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:34:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:34:55 Deploying app: TestEventingSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:34:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps stored in temp store"
}
2021/12/24 07:35:01 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/24 07:35:06 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/24 07:35:11 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/24 07:35:16 Waiting for app: TestEventingSwapRebWithOps to get deployed
2021/12/24 07:35:16 App: TestEventingSwapRebWithOps got deployed
2021/12/24 07:35:16 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 07:35:16 Starting bucket ops to source bucket
2021/12/24 07:35:16 Removing node(s): [] from the cluster
2021/12/24 07:35:21 Rebalance progress: 12.5
2021/12/24 07:35:26 Rebalance progress: 12.5
2021/12/24 07:35:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:35:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:35:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:35:31 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 07:35:31 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:35:37 Rebalance progress: 12.5
2021/12/24 07:35:42 Rebalance progress: 12.5
2021/12/24 07:35:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:35:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:35:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:35:47 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:35:52 Rebalance progress: 12.5
2021/12/24 07:35:57 Rebalance progress: 12.5
2021/12/24 07:36:02 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:36:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:36:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:36:02 Waiting for src bucket item count to get to: 200000 curr count: 67877
2021/12/24 07:36:07 Waiting for src bucket item count to get to: 200000 curr count: 67877
2021/12/24 07:36:12 Waiting for src bucket item count to get to: 200000 curr count: 82419
2021/12/24 07:36:17 Waiting for src bucket item count to get to: 200000 curr count: 82419
2021/12/24 07:36:22 Waiting for src bucket item count to get to: 200000 curr count: 98709
2021/12/24 07:36:27 Waiting for src bucket item count to get to: 200000 curr count: 98709
2021/12/24 07:36:32 Waiting for src bucket item count to get to: 200000 curr count: 114818
2021/12/24 07:36:37 Waiting for src bucket item count to get to: 200000 curr count: 114818
2021/12/24 07:36:42 Waiting for src bucket item count to get to: 200000 curr count: 131237
2021/12/24 07:36:47 Waiting for src bucket item count to get to: 200000 curr count: 131237
2021/12/24 07:36:52 Waiting for src bucket item count to get to: 200000 curr count: 147799
2021/12/24 07:36:57 Waiting for src bucket item count to get to: 200000 curr count: 147799
2021/12/24 07:37:02 Waiting for src bucket item count to get to: 200000 curr count: 147799
2021/12/24 07:37:07 Waiting for src bucket item count to get to: 200000 curr count: 164198
2021/12/24 07:37:12 Waiting for src bucket item count to get to: 200000 curr count: 164198
2021/12/24 07:37:17 Waiting for src bucket item count to get to: 200000 curr count: 179122
2021/12/24 07:37:22 Waiting for src bucket item count to get to: 200000 curr count: 179122
2021/12/24 07:37:27 Waiting for src bucket item count to get to: 200000 curr count: 195572
2021/12/24 07:37:32 Waiting for src bucket item count to get to: 200000 curr count: 195572
2021/12/24 07:37:37 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 07:37:37 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 07:37:37 Starting bucket ops to source bucket
2021/12/24 07:41:22 Waiting for dst bucket item count to get to: 0 curr count: 15000
2021/12/24 07:41:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:41:22 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 07:41:27 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/24 07:41:33 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/24 07:41:38 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/24 07:41:43 Waiting for app: TestEventingSwapRebWithOps to get un-deployed
2021/12/24 07:41:43 App: TestEventingSwapRebWithOps got un-deployed
2021/12/24 07:41:43 No eventing-con process running
2021/12/24 07:41:43 Function: TestEventingSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithOps deleting in background"
}
2021/12/24 07:41:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:41:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:41:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:41:51 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:41:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:41:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithOps (450.06s)
=== RUN   TestKVRebInWithOps
2021/12/24 07:42:06 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/24 07:42:06 App: TestKVRebInWithOps got un-deployed
2021/12/24 07:42:06 No eventing-con process running
2021/12/24 07:42:06 Function: TestKVRebInWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithOps not found"
}
2021/12/24 07:42:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:42:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:42:13 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:42:15 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:42:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:42:20 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:42:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:42:25 Deploying app: TestKVRebInWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:42:26 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps stored in temp store"
}
2021/12/24 07:42:31 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/24 07:42:36 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/24 07:42:41 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/24 07:42:46 Waiting for app: TestKVRebInWithOps to get deployed
2021/12/24 07:42:46 App: TestKVRebInWithOps got deployed
2021/12/24 07:42:46 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 07:42:46 Starting bucket ops to source bucket
2021/12/24 07:42:47 Removing node(s): [] from the cluster
2021/12/24 07:42:52 Rebalance progress: 12.5
2021/12/24 07:42:57 Rebalance progress: 12.5
2021/12/24 07:43:02 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:43:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:43:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:43:02 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:43:07 Rebalance progress: 12.5
2021/12/24 07:43:12 Rebalance progress: 12.5
2021/12/24 07:43:17 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:43:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:43:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:43:17 Waiting for src bucket item count to get to: 200000 curr count: 35860
2021/12/24 07:43:22 Waiting for src bucket item count to get to: 200000 curr count: 52274
2021/12/24 07:43:27 Waiting for src bucket item count to get to: 200000 curr count: 52274
2021/12/24 07:43:32 Waiting for src bucket item count to get to: 200000 curr count: 68621
2021/12/24 07:43:37 Waiting for src bucket item count to get to: 200000 curr count: 68621
2021/12/24 07:43:42 Waiting for src bucket item count to get to: 200000 curr count: 83866
2021/12/24 07:43:47 Waiting for src bucket item count to get to: 200000 curr count: 83866
2021/12/24 07:43:52 Waiting for src bucket item count to get to: 200000 curr count: 100329
2021/12/24 07:43:57 Waiting for src bucket item count to get to: 200000 curr count: 100329
2021/12/24 07:44:02 Waiting for src bucket item count to get to: 200000 curr count: 116894
2021/12/24 07:44:07 Waiting for src bucket item count to get to: 200000 curr count: 116894
2021/12/24 07:44:12 Waiting for src bucket item count to get to: 200000 curr count: 133422
2021/12/24 07:44:17 Waiting for src bucket item count to get to: 200000 curr count: 133422
2021/12/24 07:44:22 Waiting for src bucket item count to get to: 200000 curr count: 150034
2021/12/24 07:44:27 Waiting for src bucket item count to get to: 200000 curr count: 150034
2021/12/24 07:44:32 Waiting for src bucket item count to get to: 200000 curr count: 166369
2021/12/24 07:44:37 Waiting for src bucket item count to get to: 200000 curr count: 166369
2021/12/24 07:44:42 Waiting for src bucket item count to get to: 200000 curr count: 181691
2021/12/24 07:44:47 Waiting for src bucket item count to get to: 200000 curr count: 181691
2021/12/24 07:44:52 Waiting for src bucket item count to get to: 200000 curr count: 181691
2021/12/24 07:44:57 Waiting for src bucket item count to get to: 200000 curr count: 198287
2021/12/24 07:45:02 Waiting for src bucket item count to get to: 200000 curr count: 198287
2021/12/24 07:45:07 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 07:45:07 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 07:45:07 Starting bucket ops to source bucket
2021/12/24 07:48:53 Waiting for dst bucket item count to get to: 0 curr count: 16527
2021/12/24 07:48:53 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:48:53 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 07:48:58 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/24 07:49:03 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/24 07:49:08 Waiting for app: TestKVRebInWithOps to get un-deployed
2021/12/24 07:49:08 App: TestKVRebInWithOps got un-deployed
2021/12/24 07:49:08 No eventing-con process running
2021/12/24 07:49:09 Function: TestKVRebInWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithOps deleting in background"
}
2021/12/24 07:49:10 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:49:15 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:49:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:49:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:49:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:49:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithOps (445.87s)
=== RUN   TestKVRebOutWithOps
2021/12/24 07:49:27 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 07:49:28 Removing node(s): [] from the cluster
2021/12/24 07:49:33 Rebalance progress: 100
2021/12/24 07:49:38 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/24 07:49:38 App: TestKVRebOutWithOps got un-deployed
2021/12/24 07:49:38 No eventing-con process running
2021/12/24 07:49:38 Function: TestKVRebOutWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithOps not found"
}
2021/12/24 07:49:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:49:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:49:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:49:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:49:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:49:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:49:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:49:56 Deploying app: TestKVRebOutWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:49:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps stored in temp store"
}
2021/12/24 07:50:02 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/24 07:50:07 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/24 07:50:12 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/24 07:50:17 Waiting for app: TestKVRebOutWithOps to get deployed
2021/12/24 07:50:17 App: TestKVRebOutWithOps got deployed
2021/12/24 07:50:17 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:50:17 Starting bucket ops to source bucket
2021/12/24 07:50:22 Rebalance progress: 12.5
2021/12/24 07:50:27 Rebalance progress: 12.5
2021/12/24 07:50:32 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:50:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:50:32 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:50:32 Waiting for src bucket item count to get to: 200000 curr count: 18772
2021/12/24 07:50:37 Waiting for src bucket item count to get to: 200000 curr count: 18772
2021/12/24 07:50:42 Waiting for src bucket item count to get to: 200000 curr count: 34807
2021/12/24 07:50:47 Waiting for src bucket item count to get to: 200000 curr count: 34807
2021/12/24 07:50:52 Waiting for src bucket item count to get to: 200000 curr count: 51265
2021/12/24 07:50:57 Waiting for src bucket item count to get to: 200000 curr count: 51265
2021/12/24 07:51:02 Waiting for src bucket item count to get to: 200000 curr count: 67611
2021/12/24 07:51:07 Waiting for src bucket item count to get to: 200000 curr count: 67611
2021/12/24 07:51:12 Waiting for src bucket item count to get to: 200000 curr count: 67611
2021/12/24 07:51:17 Waiting for src bucket item count to get to: 200000 curr count: 82384
2021/12/24 07:51:22 Waiting for src bucket item count to get to: 200000 curr count: 82384
2021/12/24 07:51:27 Waiting for src bucket item count to get to: 200000 curr count: 98519
2021/12/24 07:51:32 Waiting for src bucket item count to get to: 200000 curr count: 98519
2021/12/24 07:51:38 Waiting for src bucket item count to get to: 200000 curr count: 114633
2021/12/24 07:51:43 Waiting for src bucket item count to get to: 200000 curr count: 114633
2021/12/24 07:51:48 Waiting for src bucket item count to get to: 200000 curr count: 130756
2021/12/24 07:51:53 Waiting for src bucket item count to get to: 200000 curr count: 130756
2021/12/24 07:51:58 Waiting for src bucket item count to get to: 200000 curr count: 146782
2021/12/24 07:52:03 Waiting for src bucket item count to get to: 200000 curr count: 146782
2021/12/24 07:52:08 Waiting for src bucket item count to get to: 200000 curr count: 163081
2021/12/24 07:52:13 Waiting for src bucket item count to get to: 200000 curr count: 163081
2021/12/24 07:52:18 Waiting for src bucket item count to get to: 200000 curr count: 178011
2021/12/24 07:52:23 Waiting for src bucket item count to get to: 200000 curr count: 178011
2021/12/24 07:52:28 Waiting for src bucket item count to get to: 200000 curr count: 194477
2021/12/24 07:52:33 Waiting for src bucket item count to get to: 200000 curr count: 194477
2021/12/24 07:52:38 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 07:52:38 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 07:52:38 Starting bucket ops to source bucket
2021/12/24 07:56:24 Waiting for dst bucket item count to get to: 0 curr count: 17716
2021/12/24 07:56:24 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:56:24 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 07:56:29 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/24 07:56:34 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/24 07:56:39 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/24 07:56:44 Waiting for app: TestKVRebOutWithOps to get un-deployed
2021/12/24 07:56:44 App: TestKVRebOutWithOps got un-deployed
2021/12/24 07:56:44 No eventing-con process running
2021/12/24 07:56:44 Function: TestKVRebOutWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithOps deleting in background"
}
2021/12/24 07:56:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:56:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:56:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:56:53 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:56:58 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:56:58 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithOps (455.67s)
=== RUN   TestKVSwapRebWithOps
2021/12/24 07:57:08 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/24 07:57:08 App: TestKVSwapRebWithOps got un-deployed
2021/12/24 07:57:08 No eventing-con process running
2021/12/24 07:57:08 Function: TestKVSwapRebWithOps delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithOps not found"
}
2021/12/24 07:57:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:57:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:57:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 07:57:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:57:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:57:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:57:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 07:57:26 Deploying app: TestKVSwapRebWithOps. Item count src bucket: 0 dst bucket: 0
2021/12/24 07:57:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps stored in temp store"
}
2021/12/24 07:57:32 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/24 07:57:37 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/24 07:57:42 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/24 07:57:47 Waiting for app: TestKVSwapRebWithOps to get deployed
2021/12/24 07:57:47 App: TestKVSwapRebWithOps got deployed
2021/12/24 07:57:47 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 07:57:47 Starting bucket ops to source bucket
2021/12/24 07:57:48 Removing node(s): [] from the cluster
2021/12/24 07:57:53 Rebalance progress: 12.5
2021/12/24 07:57:58 Rebalance progress: 12.5
2021/12/24 07:58:03 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:03 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:58:03 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 07:58:03 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 07:58:08 Rebalance progress: 12.5
2021/12/24 07:58:13 Rebalance progress: 12.5
2021/12/24 07:58:18 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:58:18 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 07:58:24 Rebalance progress: 12.5
2021/12/24 07:58:29 Rebalance progress: 12.5
2021/12/24 07:58:34 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 07:58:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 07:58:34 Waiting for src bucket item count to get to: 200000 curr count: 65098
2021/12/24 07:58:39 Waiting for src bucket item count to get to: 200000 curr count: 65098
2021/12/24 07:58:44 Waiting for src bucket item count to get to: 200000 curr count: 79685
2021/12/24 07:58:49 Waiting for src bucket item count to get to: 200000 curr count: 79685
2021/12/24 07:58:54 Waiting for src bucket item count to get to: 200000 curr count: 96207
2021/12/24 07:58:59 Waiting for src bucket item count to get to: 200000 curr count: 96207
2021/12/24 07:59:04 Waiting for src bucket item count to get to: 200000 curr count: 112845
2021/12/24 07:59:09 Waiting for src bucket item count to get to: 200000 curr count: 112845
2021/12/24 07:59:14 Waiting for src bucket item count to get to: 200000 curr count: 129331
2021/12/24 07:59:19 Waiting for src bucket item count to get to: 200000 curr count: 129331
2021/12/24 07:59:24 Waiting for src bucket item count to get to: 200000 curr count: 145833
2021/12/24 07:59:29 Waiting for src bucket item count to get to: 200000 curr count: 145833
2021/12/24 07:59:34 Waiting for src bucket item count to get to: 200000 curr count: 162335
2021/12/24 07:59:39 Waiting for src bucket item count to get to: 200000 curr count: 162335
2021/12/24 07:59:44 Waiting for src bucket item count to get to: 200000 curr count: 177555
2021/12/24 07:59:49 Waiting for src bucket item count to get to: 200000 curr count: 177555
2021/12/24 07:59:54 Waiting for src bucket item count to get to: 200000 curr count: 177555
2021/12/24 07:59:59 Waiting for src bucket item count to get to: 200000 curr count: 193800
2021/12/24 08:00:04 Waiting for src bucket item count to get to: 200000 curr count: 193800
2021/12/24 08:00:09 src & dst bucket item count matched up. src bucket count: 200000 dst bucket count: 200000
2021/12/24 08:00:09 Post SET ops. Expected item count: 200000 got 200000
2021/12/24 08:00:09 Starting bucket ops to source bucket
2021/12/24 08:03:54 Waiting for dst bucket item count to get to: 0 curr count: 18928
2021/12/24 08:03:59 Waiting for dst bucket item count to get to: 0 curr count: 152
2021/12/24 08:04:04 Waiting for dst bucket item count to get to: 0 curr count: 152
2021/12/24 08:04:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:04:04 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:04:10 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/24 08:04:15 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/24 08:04:20 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/24 08:04:25 Waiting for app: TestKVSwapRebWithOps to get un-deployed
2021/12/24 08:04:25 App: TestKVSwapRebWithOps got un-deployed
2021/12/24 08:04:25 No eventing-con process running
2021/12/24 08:04:25 Function: TestKVSwapRebWithOps delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithOps deleting in background"
}
2021/12/24 08:04:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:04:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:04:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:04:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:04:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:04:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithOps (460.51s)
=== RUN   TestEventingRebInWithTimers
2021/12/24 08:04:48 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/24 08:04:48 App: TestEventingRebInWithTimers got un-deployed
2021/12/24 08:04:48 No eventing-con process running
2021/12/24 08:04:48 Function: TestEventingRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebInWithTimers not found"
}
2021/12/24 08:04:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:04:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:04:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:04:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:05:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:05:07 Deploying app: TestEventingRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:05:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers stored in temp store"
}
2021/12/24 08:05:13 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/24 08:05:18 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/24 08:05:23 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/24 08:05:28 Waiting for app: TestEventingRebInWithTimers to get deployed
2021/12/24 08:05:28 App: TestEventingRebInWithTimers got deployed
2021/12/24 08:05:28 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:05:28 Starting bucket ops to source bucket
2021/12/24 08:05:28 Removing node(s): [] from the cluster
2021/12/24 08:05:33 Rebalance progress: 12.5
2021/12/24 08:05:38 Rebalance progress: 12.5
2021/12/24 08:05:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:05:43 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:05:49 Rebalance progress: 12.5
2021/12/24 08:05:54 Rebalance progress: 12.5
2021/12/24 08:05:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:05:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:05:59 Waiting for src bucket item count to get to: 50000 curr count: 27110
2021/12/24 08:06:04 Waiting for src bucket item count to get to: 50000 curr count: 37064
2021/12/24 08:06:09 Waiting for src bucket item count to get to: 50000 curr count: 37064
2021/12/24 08:06:14 Waiting for src bucket item count to get to: 50000 curr count: 49762
2021/12/24 08:06:19 Waiting for src bucket item count to get to: 50000 curr count: 49762
2021/12/24 08:06:29 Waiting for dst bucket item count to get to: 50000 curr count: 47107
2021/12/24 08:06:34 Waiting for dst bucket item count to get to: 50000 curr count: 47107
2021/12/24 08:06:34 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:06:34 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:06:34 Starting bucket ops to source bucket
2021/12/24 08:07:43 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:07:48 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:07:53 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:07:58 Waiting for dst bucket item count to get to: 0 curr count: 40878
2021/12/24 08:08:03 Waiting for dst bucket item count to get to: 0 curr count: 40878
2021/12/24 08:08:08 Waiting for dst bucket item count to get to: 0 curr count: 27238
2021/12/24 08:08:13 Waiting for dst bucket item count to get to: 0 curr count: 27238
2021/12/24 08:08:18 Waiting for dst bucket item count to get to: 0 curr count: 10703
2021/12/24 08:08:23 Waiting for dst bucket item count to get to: 0 curr count: 10703
2021/12/24 08:08:23 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:08:23 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:08:28 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/24 08:08:34 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/24 08:08:39 Waiting for app: TestEventingRebInWithTimers to get un-deployed
2021/12/24 08:08:39 App: TestEventingRebInWithTimers got un-deployed
2021/12/24 08:08:39 No eventing-con process running
2021/12/24 08:08:39 Function: TestEventingRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebInWithTimers deleting in background"
}
2021/12/24 08:08:40 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:08:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:08:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:08:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:08:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:08:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebInWithTimers (253.84s)
=== RUN   TestEventingRebOutWithTimers
2021/12/24 08:08:57 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:08:58 Removing node(s): [] from the cluster
2021/12/24 08:09:03 Rebalance progress: 100
2021/12/24 08:09:08 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/24 08:09:08 App: TestEventingRebOutWithTimers got un-deployed
2021/12/24 08:09:08 No eventing-con process running
2021/12/24 08:09:08 Function: TestEventingRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingRebOutWithTimers not found"
}
2021/12/24 08:09:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:09:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:09:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:09:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:09:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:09:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:09:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:09:26 Deploying app: TestEventingRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:09:27 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers stored in temp store"
}
2021/12/24 08:09:32 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/24 08:09:37 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/24 08:09:42 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/24 08:09:47 Waiting for app: TestEventingRebOutWithTimers to get deployed
2021/12/24 08:09:47 App: TestEventingRebOutWithTimers got deployed
2021/12/24 08:09:47 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:09:47 Starting bucket ops to source bucket
2021/12/24 08:09:52 Rebalance progress: 12.5
2021/12/24 08:09:57 Rebalance progress: 12.5
2021/12/24 08:10:02 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:10:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:10:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:10:02 Waiting for src bucket item count to get to: 50000 curr count: 16854
2021/12/24 08:10:07 Waiting for src bucket item count to get to: 50000 curr count: 16854
2021/12/24 08:10:12 Waiting for src bucket item count to get to: 50000 curr count: 28377
2021/12/24 08:10:17 Waiting for src bucket item count to get to: 50000 curr count: 28377
2021/12/24 08:10:22 Waiting for src bucket item count to get to: 50000 curr count: 38932
2021/12/24 08:10:27 Waiting for src bucket item count to get to: 50000 curr count: 38932
2021/12/24 08:10:37 Waiting for dst bucket item count to get to: 50000 curr count: 34750
2021/12/24 08:10:42 Waiting for dst bucket item count to get to: 50000 curr count: 34750
2021/12/24 08:10:47 Waiting for dst bucket item count to get to: 50000 curr count: 34750
2021/12/24 08:10:52 Waiting for dst bucket item count to get to: 50000 curr count: 47513
2021/12/24 08:10:57 Waiting for dst bucket item count to get to: 50000 curr count: 47513
2021/12/24 08:10:57 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:10:57 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:10:57 Starting bucket ops to source bucket
2021/12/24 08:12:07 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:12:12 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:12:17 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:12:22 Waiting for dst bucket item count to get to: 0 curr count: 47387
2021/12/24 08:12:27 Waiting for dst bucket item count to get to: 0 curr count: 47387
2021/12/24 08:12:32 Waiting for dst bucket item count to get to: 0 curr count: 37768
2021/12/24 08:12:37 Waiting for dst bucket item count to get to: 0 curr count: 37768
2021/12/24 08:12:42 Waiting for dst bucket item count to get to: 0 curr count: 15808
2021/12/24 08:12:47 Waiting for dst bucket item count to get to: 0 curr count: 15808
2021/12/24 08:12:47 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:12:47 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:12:52 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/24 08:12:57 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/24 08:13:02 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/24 08:13:07 Waiting for app: TestEventingRebOutWithTimers to get un-deployed
2021/12/24 08:13:07 App: TestEventingRebOutWithTimers got un-deployed
2021/12/24 08:13:07 No eventing-con process running
2021/12/24 08:13:07 Function: TestEventingRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingRebOutWithTimers deleting in background"
}
2021/12/24 08:13:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:13:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:13:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:13:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:13:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:13:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingRebOutWithTimers (268.69s)
=== RUN   TestEventingSwapRebWithTimers
2021/12/24 08:13:31 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/24 08:13:31 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/24 08:13:31 No eventing-con process running
2021/12/24 08:13:31 Function: TestEventingSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestEventingSwapRebWithTimers not found"
}
2021/12/24 08:13:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:13:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:13:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:13:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:13:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:13:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:13:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:13:49 Deploying app: TestEventingSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:13:50 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers stored in temp store"
}
2021/12/24 08:13:55 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/24 08:14:00 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/24 08:14:05 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/24 08:14:11 Waiting for app: TestEventingSwapRebWithTimers to get deployed
2021/12/24 08:14:11 App: TestEventingSwapRebWithTimers got deployed
2021/12/24 08:14:11 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:14:11 Starting bucket ops to source bucket
2021/12/24 08:14:11 Removing node(s): [] from the cluster
2021/12/24 08:14:16 Rebalance progress: 12.5
2021/12/24 08:14:21 Rebalance progress: 12.5
2021/12/24 08:14:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:14:26 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 08:14:27 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:14:32 Rebalance progress: 12.5
2021/12/24 08:14:37 Rebalance progress: 12.5
2021/12/24 08:14:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:14:42 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 08:14:47 Rebalance progress: 12.5
2021/12/24 08:14:52 Rebalance progress: 12.5
2021/12/24 08:14:57 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:14:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:14:57 Waiting for src bucket item count to get to: 50000 curr count: 44271
2021/12/24 08:15:07 Waiting for dst bucket item count to get to: 50000 curr count: 41961
2021/12/24 08:15:12 Waiting for dst bucket item count to get to: 50000 curr count: 41961
2021/12/24 08:15:12 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:15:12 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:15:12 Starting bucket ops to source bucket
2021/12/24 08:16:21 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:16:26 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:16:31 Waiting for dst bucket item count to get to: 0 curr count: 47428
2021/12/24 08:16:36 Waiting for dst bucket item count to get to: 0 curr count: 47428
2021/12/24 08:16:41 Waiting for dst bucket item count to get to: 0 curr count: 36996
2021/12/24 08:16:46 Waiting for dst bucket item count to get to: 0 curr count: 36996
2021/12/24 08:16:51 Waiting for dst bucket item count to get to: 0 curr count: 16451
2021/12/24 08:16:56 Waiting for dst bucket item count to get to: 0 curr count: 16451
2021/12/24 08:17:01 Waiting for dst bucket item count to get to: 0 curr count: 2642
2021/12/24 08:17:06 Waiting for dst bucket item count to get to: 0 curr count: 2642
2021/12/24 08:17:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:17:06 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:17:11 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/24 08:17:16 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/24 08:17:22 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/24 08:17:27 Waiting for app: TestEventingSwapRebWithTimers to get un-deployed
2021/12/24 08:17:27 App: TestEventingSwapRebWithTimers got un-deployed
2021/12/24 08:17:27 No eventing-con process running
2021/12/24 08:17:27 Function: TestEventingSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestEventingSwapRebWithTimers deleting in background"
}
2021/12/24 08:17:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:17:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:17:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:17:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:17:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:17:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestEventingSwapRebWithTimers (259.46s)
=== RUN   TestKVRebInWithTimers
2021/12/24 08:17:50 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/24 08:17:50 App: TestKVRebInWithTimers got un-deployed
2021/12/24 08:17:50 No eventing-con process running
2021/12/24 08:17:50 Function: TestKVRebInWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebInWithTimers not found"
}
2021/12/24 08:17:52 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:17:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:17:57 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:17:59 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:18:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:18:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:18:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:18:09 Deploying app: TestKVRebInWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:18:10 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers stored in temp store"
}
2021/12/24 08:18:15 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/24 08:18:20 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/24 08:18:25 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/24 08:18:30 Waiting for app: TestKVRebInWithTimers to get deployed
2021/12/24 08:18:30 App: TestKVRebInWithTimers got deployed
2021/12/24 08:18:30 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:18:30 Starting bucket ops to source bucket
2021/12/24 08:18:30 Removing node(s): [] from the cluster
2021/12/24 08:18:35 Rebalance progress: 12.5
2021/12/24 08:18:40 Rebalance progress: 12.5
2021/12/24 08:18:45 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:18:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:18:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:18:45 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:18:50 Rebalance progress: 12.5
2021/12/24 08:18:55 Rebalance progress: 12.5
2021/12/24 08:19:00 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:19:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:19:00 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:19:01 Waiting for src bucket item count to get to: 50000 curr count: 34060
2021/12/24 08:19:06 Waiting for src bucket item count to get to: 50000 curr count: 34060
2021/12/24 08:19:11 Waiting for src bucket item count to get to: 50000 curr count: 46247
2021/12/24 08:19:16 Waiting for src bucket item count to get to: 50000 curr count: 46247
2021/12/24 08:19:26 Waiting for dst bucket item count to get to: 50000 curr count: 45393
2021/12/24 08:19:31 Waiting for dst bucket item count to get to: 50000 curr count: 45393
2021/12/24 08:19:31 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:19:31 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:19:31 Starting bucket ops to source bucket
2021/12/24 08:20:40 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:20:45 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:20:50 Waiting for dst bucket item count to get to: 0 curr count: 47669
2021/12/24 08:20:55 Waiting for dst bucket item count to get to: 0 curr count: 47669
2021/12/24 08:21:00 Waiting for dst bucket item count to get to: 0 curr count: 36833
2021/12/24 08:21:05 Waiting for dst bucket item count to get to: 0 curr count: 36833
2021/12/24 08:21:10 Waiting for dst bucket item count to get to: 0 curr count: 14791
2021/12/24 08:21:15 Waiting for dst bucket item count to get to: 0 curr count: 14791
2021/12/24 08:21:20 Waiting for dst bucket item count to get to: 0 curr count: 2855
2021/12/24 08:21:25 Waiting for dst bucket item count to get to: 0 curr count: 2855
2021/12/24 08:21:25 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:21:25 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:21:30 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/24 08:21:35 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/24 08:21:41 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/24 08:21:46 Waiting for app: TestKVRebInWithTimers to get un-deployed
2021/12/24 08:21:46 App: TestKVRebInWithTimers got un-deployed
2021/12/24 08:21:46 No eventing-con process running
2021/12/24 08:21:46 Function: TestKVRebInWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebInWithTimers deleting in background"
}
2021/12/24 08:21:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:21:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:21:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:21:54 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:21:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:21:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebInWithTimers (259.02s)
=== RUN   TestKVRebOutWithTimers
2021/12/24 08:22:04 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:22:04 Removing node(s): [] from the cluster
2021/12/24 08:22:09 Rebalance progress: 100
2021/12/24 08:22:14 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/24 08:22:14 App: TestKVRebOutWithTimers got un-deployed
2021/12/24 08:22:14 No eventing-con process running
2021/12/24 08:22:15 Function: TestKVRebOutWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVRebOutWithTimers not found"
}
2021/12/24 08:22:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:22:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:22:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:22:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:22:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:22:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:22:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:22:33 Deploying app: TestKVRebOutWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:22:34 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers stored in temp store"
}
2021/12/24 08:22:39 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/24 08:22:44 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/24 08:22:49 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/24 08:22:54 Waiting for app: TestKVRebOutWithTimers to get deployed
2021/12/24 08:22:54 App: TestKVRebOutWithTimers got deployed
2021/12/24 08:22:54 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:22:54 Starting bucket ops to source bucket
2021/12/24 08:22:59 Rebalance progress: 12.5
2021/12/24 08:23:04 Rebalance progress: 12.5
2021/12/24 08:23:09 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:23:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:23:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:23:09 Waiting for src bucket item count to get to: 50000 curr count: 6651
2021/12/24 08:23:14 Waiting for src bucket item count to get to: 50000 curr count: 20677
2021/12/24 08:23:19 Waiting for src bucket item count to get to: 50000 curr count: 20677
2021/12/24 08:23:24 Waiting for src bucket item count to get to: 50000 curr count: 32075
2021/12/24 08:23:29 Waiting for src bucket item count to get to: 50000 curr count: 32075
2021/12/24 08:23:34 Waiting for src bucket item count to get to: 50000 curr count: 43283
2021/12/24 08:23:39 Waiting for src bucket item count to get to: 50000 curr count: 43283
2021/12/24 08:23:44 Waiting for src bucket item count to get to: 50000 curr count: 43283
2021/12/24 08:23:54 Waiting for dst bucket item count to get to: 50000 curr count: 35142
2021/12/24 08:23:59 Waiting for dst bucket item count to get to: 50000 curr count: 35142
2021/12/24 08:24:04 Waiting for dst bucket item count to get to: 50000 curr count: 42658
2021/12/24 08:24:04 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:24:04 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:24:04 Starting bucket ops to source bucket
2021/12/24 08:25:14 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:25:19 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:25:24 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:25:29 Waiting for dst bucket item count to get to: 0 curr count: 47278
2021/12/24 08:25:34 Waiting for dst bucket item count to get to: 0 curr count: 47278
2021/12/24 08:25:39 Waiting for dst bucket item count to get to: 0 curr count: 30029
2021/12/24 08:25:44 Waiting for dst bucket item count to get to: 0 curr count: 30029
2021/12/24 08:25:49 Waiting for dst bucket item count to get to: 0 curr count: 16668
2021/12/24 08:25:54 Waiting for dst bucket item count to get to: 0 curr count: 16668
2021/12/24 08:25:59 Waiting for dst bucket item count to get to: 0 curr count: 1111
2021/12/24 08:26:04 Waiting for dst bucket item count to get to: 0 curr count: 1111
2021/12/24 08:26:04 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:26:04 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:26:09 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/24 08:26:14 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/24 08:26:20 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/24 08:26:25 Waiting for app: TestKVRebOutWithTimers to get un-deployed
2021/12/24 08:26:25 App: TestKVRebOutWithTimers got un-deployed
2021/12/24 08:26:25 No eventing-con process running
2021/12/24 08:26:25 Function: TestKVRebOutWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVRebOutWithTimers deleting in background"
}
2021/12/24 08:26:26 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:26:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:26:31 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:26:33 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:26:38 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:26:38 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOutWithTimers (278.91s)
=== RUN   TestKVSwapRebWithTimers
2021/12/24 08:26:48 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:26:48 App: TestKVSwapRebWithTimers got un-deployed
2021/12/24 08:26:48 No eventing-con process running
2021/12/24 08:26:48 Function: TestKVSwapRebWithTimers delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestKVSwapRebWithTimers not found"
}
2021/12/24 08:26:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:26:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:26:55 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:26:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:27:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:27:07 Deploying app: TestKVSwapRebWithTimers. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:27:08 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers stored in temp store"
}
2021/12/24 08:27:13 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/24 08:27:18 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/24 08:27:23 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/24 08:27:28 Waiting for app: TestKVSwapRebWithTimers to get deployed
2021/12/24 08:27:28 App: TestKVSwapRebWithTimers got deployed
2021/12/24 08:27:28 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:27:28 Starting bucket ops to source bucket
2021/12/24 08:27:28 Removing node(s): [] from the cluster
2021/12/24 08:27:33 Rebalance progress: 12.5
2021/12/24 08:27:38 Rebalance progress: 12.5
2021/12/24 08:27:43 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:27:43 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 08:27:44 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:27:49 Rebalance progress: 12.5
2021/12/24 08:27:54 Rebalance progress: 12.5
2021/12/24 08:27:59 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:27:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:27:59 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 08:28:04 Rebalance progress: 12.5
2021/12/24 08:28:09 Rebalance progress: 12.5
2021/12/24 08:28:14 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:28:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:28:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:28:14 Waiting for src bucket item count to get to: 50000 curr count: 49663
2021/12/24 08:28:19 Waiting for src bucket item count to get to: 50000 curr count: 49663
2021/12/24 08:28:29 Waiting for dst bucket item count to get to: 50000 curr count: 41110
2021/12/24 08:28:34 Waiting for dst bucket item count to get to: 50000 curr count: 41110
2021/12/24 08:28:39 Waiting for dst bucket item count to get to: 50000 curr count: 49499
2021/12/24 08:28:44 Waiting for dst bucket item count to get to: 50000 curr count: 49499
2021/12/24 08:28:44 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:28:44 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:28:44 Starting bucket ops to source bucket
2021/12/24 08:29:53 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:29:58 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:30:03 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:30:08 Waiting for dst bucket item count to get to: 0 curr count: 47299
2021/12/24 08:30:13 Waiting for dst bucket item count to get to: 0 curr count: 47299
2021/12/24 08:30:18 Waiting for dst bucket item count to get to: 0 curr count: 31943
2021/12/24 08:30:23 Waiting for dst bucket item count to get to: 0 curr count: 31943
2021/12/24 08:30:28 Waiting for dst bucket item count to get to: 0 curr count: 14919
2021/12/24 08:30:33 Waiting for dst bucket item count to get to: 0 curr count: 14919
2021/12/24 08:30:38 Waiting for dst bucket item count to get to: 0 curr count: 1832
2021/12/24 08:30:43 Waiting for dst bucket item count to get to: 0 curr count: 1832
2021/12/24 08:30:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:30:43 Post DELETE ops. Expected item count: 0 got 0
2021/12/24 08:30:48 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:30:53 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:30:59 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:31:04 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:31:09 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:31:14 Waiting for app: TestKVSwapRebWithTimers to get un-deployed
2021/12/24 08:31:14 App: TestKVSwapRebWithTimers got un-deployed
2021/12/24 08:31:14 No eventing-con process running
2021/12/24 08:31:14 Function: TestKVSwapRebWithTimers delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestKVSwapRebWithTimers deleting in background"
}
2021/12/24 08:31:15 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:31:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:31:21 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:31:22 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:31:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:31:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebWithTimers (289.14s)
=== RUN   TestDuplicateTimerCreationEventingRebIn
2021/12/24 08:31:37 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:31:37 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/24 08:31:37 No eventing-con process running
2021/12/24 08:31:37 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebIn not found"
}
2021/12/24 08:31:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:31:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:31:44 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:31:46 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:31:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:31:51 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:31:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:31:56 Deploying app: TestDuplicateTimerCreationEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:31:57 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn stored in temp store"
}
2021/12/24 08:32:02 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/24 08:32:07 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/24 08:32:12 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/24 08:32:17 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get deployed
2021/12/24 08:32:17 App: TestDuplicateTimerCreationEventingRebIn got deployed
2021/12/24 08:32:17 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:32:17 Starting bucket ops to source bucket
2021/12/24 08:32:18 Removing node(s): [] from the cluster
2021/12/24 08:32:23 Rebalance progress: 12.5
2021/12/24 08:32:28 Rebalance progress: 12.5
2021/12/24 08:32:33 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:32:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:32:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:32:33 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:32:38 Rebalance progress: 12.5
2021/12/24 08:32:43 Rebalance progress: 12.5
2021/12/24 08:32:48 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:32:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:32:48 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:32:48 Waiting for src bucket item count to get to: 50000 curr count: 28405
2021/12/24 08:32:53 Waiting for src bucket item count to get to: 50000 curr count: 39267
2021/12/24 08:32:58 Waiting for src bucket item count to get to: 50000 curr count: 39267
2021/12/24 08:33:08 Waiting for dst bucket item count to get to: 50000 curr count: 27670
2021/12/24 08:33:13 Waiting for dst bucket item count to get to: 50000 curr count: 27670
2021/12/24 08:33:18 Waiting for dst bucket item count to get to: 50000 curr count: 39493
2021/12/24 08:33:23 Waiting for dst bucket item count to get to: 50000 curr count: 39493
2021/12/24 08:33:28 Waiting for dst bucket item count to get to: 50000 curr count: 47151
2021/12/24 08:33:33 Waiting for dst bucket item count to get to: 50000 curr count: 47151
2021/12/24 08:33:33 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:33:33 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:33:38 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:33:43 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:33:48 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:33:53 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:33:58 Waiting for app: TestDuplicateTimerCreationEventingRebIn to get un-deployed
2021/12/24 08:33:58 App: TestDuplicateTimerCreationEventingRebIn got un-deployed
2021/12/24 08:33:58 No eventing-con process running
2021/12/24 08:33:58 Function: TestDuplicateTimerCreationEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebIn deleting in background"
}
2021/12/24 08:34:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:34:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:34:10 Waiting for dst bucket item count to get to: 0 curr count: 7338
2021/12/24 08:34:15 Waiting for dst bucket item count to get to: 0 curr count: 7338
2021/12/24 08:34:15 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:34:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:34:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:34:27 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:34:27 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebIn (179.55s)
=== RUN   TestDuplicateTimerCreationEventingRebOut
2021/12/24 08:34:32 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:34:32 Removing node(s): [] from the cluster
2021/12/24 08:34:37 Rebalance progress: 100
2021/12/24 08:34:42 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:34:42 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/24 08:34:42 No eventing-con process running
2021/12/24 08:34:42 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingRebOut not found"
}
2021/12/24 08:34:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:34:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:34:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:34:51 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:34:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:34:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:35:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:35:01 Deploying app: TestDuplicateTimerCreationEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:35:02 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut stored in temp store"
}
2021/12/24 08:35:07 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/24 08:35:12 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/24 08:35:17 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/24 08:35:22 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get deployed
2021/12/24 08:35:22 App: TestDuplicateTimerCreationEventingRebOut got deployed
2021/12/24 08:35:22 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:35:22 Starting bucket ops to source bucket
2021/12/24 08:35:27 Rebalance progress: 12.5
2021/12/24 08:35:32 Rebalance progress: 12.5
2021/12/24 08:35:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:35:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:35:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:35:37 Waiting for src bucket item count to get to: 50000 curr count: 10115
2021/12/24 08:35:42 Waiting for src bucket item count to get to: 50000 curr count: 24645
2021/12/24 08:35:47 Waiting for src bucket item count to get to: 50000 curr count: 24645
2021/12/24 08:35:52 Waiting for src bucket item count to get to: 50000 curr count: 38231
2021/12/24 08:35:57 Waiting for src bucket item count to get to: 50000 curr count: 38231
2021/12/24 08:36:02 Waiting for src bucket item count to get to: 50000 curr count: 47199
2021/12/24 08:36:07 Waiting for src bucket item count to get to: 50000 curr count: 47199
2021/12/24 08:36:17 Waiting for dst bucket item count to get to: 50000 curr count: 31612
2021/12/24 08:36:22 Waiting for dst bucket item count to get to: 50000 curr count: 31612
2021/12/24 08:36:27 Waiting for dst bucket item count to get to: 50000 curr count: 31612
2021/12/24 08:36:32 Waiting for dst bucket item count to get to: 50000 curr count: 44947
2021/12/24 08:36:37 Waiting for dst bucket item count to get to: 50000 curr count: 44947
2021/12/24 08:36:37 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:36:37 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:36:42 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:36:47 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:36:52 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:36:58 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:37:03 Waiting for app: TestDuplicateTimerCreationEventingRebOut to get un-deployed
2021/12/24 08:37:03 App: TestDuplicateTimerCreationEventingRebOut got un-deployed
2021/12/24 08:37:03 No eventing-con process running
2021/12/24 08:37:03 Function: TestDuplicateTimerCreationEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingRebOut deleting in background"
}
2021/12/24 08:37:04 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:37:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:37:14 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:37:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:37:16 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:37:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:37:26 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:37:26 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingRebOut (179.40s)
=== RUN   TestDuplicateTimerCreationEventingSwapReb
2021/12/24 08:37:36 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:37:36 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/24 08:37:36 No eventing-con process running
2021/12/24 08:37:36 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationEventingSwapReb not found"
}
2021/12/24 08:37:38 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:37:43 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:37:43 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:37:45 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:37:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:37:50 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:37:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:37:55 Deploying app: TestDuplicateTimerCreationEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:37:56 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb stored in temp store"
}
2021/12/24 08:38:01 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/24 08:38:06 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/24 08:38:11 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/24 08:38:16 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get deployed
2021/12/24 08:38:16 App: TestDuplicateTimerCreationEventingSwapReb got deployed
2021/12/24 08:38:16 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:38:16 Starting bucket ops to source bucket
2021/12/24 08:38:16 Removing node(s): [] from the cluster
2021/12/24 08:38:21 Rebalance progress: 12.5
2021/12/24 08:38:26 Rebalance progress: 12.5
2021/12/24 08:38:31 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:38:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:38:31 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:38:31 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 08:38:32 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:38:37 Rebalance progress: 12.5
2021/12/24 08:38:42 Rebalance progress: 12.5
2021/12/24 08:38:47 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:38:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:38:47 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:38:47 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 08:38:52 Rebalance progress: 12.5
2021/12/24 08:38:57 Rebalance progress: 12.5
2021/12/24 08:39:02 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:39:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:39:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:39:08 Waiting for dst bucket item count to get to: 50000 curr count: 28740
2021/12/24 08:39:13 Waiting for dst bucket item count to get to: 50000 curr count: 28740
2021/12/24 08:39:18 Waiting for dst bucket item count to get to: 50000 curr count: 36096
2021/12/24 08:39:23 Waiting for dst bucket item count to get to: 50000 curr count: 36096
2021/12/24 08:39:28 Waiting for dst bucket item count to get to: 50000 curr count: 46126
2021/12/24 08:39:33 Waiting for dst bucket item count to get to: 50000 curr count: 46126
2021/12/24 08:39:33 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:39:33 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:39:38 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:39:43 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:39:48 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:39:53 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:39:58 Waiting for app: TestDuplicateTimerCreationEventingSwapReb to get un-deployed
2021/12/24 08:39:58 App: TestDuplicateTimerCreationEventingSwapReb got un-deployed
2021/12/24 08:39:58 No eventing-con process running
2021/12/24 08:39:58 Function: TestDuplicateTimerCreationEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationEventingSwapReb deleting in background"
}
2021/12/24 08:40:00 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:40:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:40:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:40:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:40:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:40:17 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:40:17 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationEventingSwapReb (170.40s)
=== RUN   TestDuplicateTimerCreationKVRebIn
2021/12/24 08:40:27 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:40:27 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/24 08:40:27 No eventing-con process running
2021/12/24 08:40:27 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebIn not found"
}
2021/12/24 08:40:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:40:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:40:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:40:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:40:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:40:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:40:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:40:45 Deploying app: TestDuplicateTimerCreationKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:40:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn stored in temp store"
}
2021/12/24 08:40:51 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/24 08:40:56 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/24 08:41:01 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/24 08:41:06 Waiting for app: TestDuplicateTimerCreationKVRebIn to get deployed
2021/12/24 08:41:06 App: TestDuplicateTimerCreationKVRebIn got deployed
2021/12/24 08:41:06 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:41:06 Starting bucket ops to source bucket
2021/12/24 08:41:07 Removing node(s): [] from the cluster
2021/12/24 08:41:12 Rebalance progress: 12.5
2021/12/24 08:41:17 Rebalance progress: 12.5
2021/12/24 08:41:22 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:41:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:41:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:41:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:41:27 Rebalance progress: 12.5
2021/12/24 08:41:32 Rebalance progress: 12.5
2021/12/24 08:41:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:41:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:41:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:41:37 Waiting for src bucket item count to get to: 50000 curr count: 30255
2021/12/24 08:41:42 Waiting for src bucket item count to get to: 50000 curr count: 41509
2021/12/24 08:41:47 Waiting for src bucket item count to get to: 50000 curr count: 41509
2021/12/24 08:41:57 Waiting for dst bucket item count to get to: 50000 curr count: 25368
2021/12/24 08:42:02 Waiting for dst bucket item count to get to: 50000 curr count: 25368
2021/12/24 08:42:07 Waiting for dst bucket item count to get to: 50000 curr count: 36379
2021/12/24 08:42:12 Waiting for dst bucket item count to get to: 50000 curr count: 36379
2021/12/24 08:42:17 Waiting for dst bucket item count to get to: 50000 curr count: 46136
2021/12/24 08:42:22 Waiting for dst bucket item count to get to: 50000 curr count: 46136
2021/12/24 08:42:22 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:42:22 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:42:27 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:42:32 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:42:37 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:42:43 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:42:48 Waiting for app: TestDuplicateTimerCreationKVRebIn to get un-deployed
2021/12/24 08:42:48 App: TestDuplicateTimerCreationKVRebIn got un-deployed
2021/12/24 08:42:48 No eventing-con process running
2021/12/24 08:42:48 Function: TestDuplicateTimerCreationKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebIn deleting in background"
}
2021/12/24 08:42:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:42:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:42:54 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:42:56 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:43:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:43:06 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:43:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebIn (169.58s)
=== RUN   TestDuplicateTimerCreationKVRebOut
2021/12/24 08:43:11 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:43:12 Removing node(s): [] from the cluster
2021/12/24 08:43:17 Rebalance progress: 100
2021/12/24 08:43:22 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:43:22 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/24 08:43:22 No eventing-con process running
2021/12/24 08:43:22 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVRebOut not found"
}
2021/12/24 08:43:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:43:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:43:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:43:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:43:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:43:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:43:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:43:40 Deploying app: TestDuplicateTimerCreationKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:43:41 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut stored in temp store"
}
2021/12/24 08:43:46 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/24 08:43:51 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/24 08:43:56 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/24 08:44:01 Waiting for app: TestDuplicateTimerCreationKVRebOut to get deployed
2021/12/24 08:44:01 App: TestDuplicateTimerCreationKVRebOut got deployed
2021/12/24 08:44:01 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:44:01 Starting bucket ops to source bucket
2021/12/24 08:44:07 Rebalance progress: 12.5
2021/12/24 08:44:12 Rebalance progress: 12.5
2021/12/24 08:44:17 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:44:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:44:17 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:44:17 Waiting for src bucket item count to get to: 50000 curr count: 10123
2021/12/24 08:44:22 Waiting for src bucket item count to get to: 50000 curr count: 24080
2021/12/24 08:44:27 Waiting for src bucket item count to get to: 50000 curr count: 24080
2021/12/24 08:44:32 Waiting for src bucket item count to get to: 50000 curr count: 38086
2021/12/24 08:44:37 Waiting for src bucket item count to get to: 50000 curr count: 38086
2021/12/24 08:44:42 Waiting for src bucket item count to get to: 50000 curr count: 47412
2021/12/24 08:44:47 Waiting for src bucket item count to get to: 50000 curr count: 47412
2021/12/24 08:44:57 Waiting for dst bucket item count to get to: 50000 curr count: 28655
2021/12/24 08:45:02 Waiting for dst bucket item count to get to: 50000 curr count: 28655
2021/12/24 08:45:07 Waiting for dst bucket item count to get to: 50000 curr count: 28655
2021/12/24 08:45:12 Waiting for dst bucket item count to get to: 50000 curr count: 44908
2021/12/24 08:45:17 Waiting for dst bucket item count to get to: 50000 curr count: 44908
2021/12/24 08:45:17 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:45:17 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:45:22 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:27 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:32 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:37 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:42 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:47 Waiting for app: TestDuplicateTimerCreationKVRebOut to get un-deployed
2021/12/24 08:45:47 App: TestDuplicateTimerCreationKVRebOut got un-deployed
2021/12/24 08:45:47 No eventing-con process running
2021/12/24 08:45:47 Function: TestDuplicateTimerCreationKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVRebOut deleting in background"
}
2021/12/24 08:45:49 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:45:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:45:59 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:45:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:46:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:46:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:46:11 Waiting for dst bucket item count to get to: 0 curr count: 41593
2021/12/24 08:46:16 Waiting for dst bucket item count to get to: 0 curr count: 41593
2021/12/24 08:46:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVRebOut (189.64s)
=== RUN   TestDuplicateTimerCreationKVSwapReb
2021/12/24 08:46:26 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:46:26 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/24 08:46:26 No eventing-con process running
2021/12/24 08:46:26 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerCreationKVSwapReb not found"
}
2021/12/24 08:46:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:46:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:46:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:46:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:46:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:46:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:46:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:46:44 Deploying app: TestDuplicateTimerCreationKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:46:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb stored in temp store"
}
2021/12/24 08:46:51 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/24 08:46:56 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/24 08:47:01 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/24 08:47:06 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get deployed
2021/12/24 08:47:06 App: TestDuplicateTimerCreationKVSwapReb got deployed
2021/12/24 08:47:06 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:47:06 Starting bucket ops to source bucket
2021/12/24 08:47:06 Removing node(s): [] from the cluster
2021/12/24 08:47:11 Rebalance progress: 12.5
2021/12/24 08:47:16 Rebalance progress: 12.5
2021/12/24 08:47:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:47:21 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 08:47:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:47:27 Rebalance progress: 12.5
2021/12/24 08:47:32 Rebalance progress: 12.5
2021/12/24 08:47:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:47:37 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 08:47:42 Rebalance progress: 12.5
2021/12/24 08:47:47 Rebalance progress: 12.5
2021/12/24 08:47:52 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:47:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:47:52 Waiting for src bucket item count to get to: 50000 curr count: 48353
2021/12/24 08:47:57 Waiting for src bucket item count to get to: 50000 curr count: 48353
2021/12/24 08:48:07 Waiting for dst bucket item count to get to: 50000 curr count: 37378
2021/12/24 08:48:12 Waiting for dst bucket item count to get to: 50000 curr count: 37378
2021/12/24 08:48:12 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:48:12 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:48:17 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:22 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:27 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:32 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:37 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:42 Waiting for app: TestDuplicateTimerCreationKVSwapReb to get un-deployed
2021/12/24 08:48:42 App: TestDuplicateTimerCreationKVSwapReb got un-deployed
2021/12/24 08:48:42 No eventing-con process running
2021/12/24 08:48:42 Function: TestDuplicateTimerCreationKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerCreationKVSwapReb deleting in background"
}
2021/12/24 08:48:44 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:48:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:48:54 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:48:59 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:48:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:49:01 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:49:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:49:11 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:49:16 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:49:16 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerCreationKVSwapReb (180.22s)
=== RUN   TestDuplicateTimerFiringEventingRebIn
2021/12/24 08:49:26 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:49:26 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/24 08:49:26 No eventing-con process running
2021/12/24 08:49:26 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebIn not found"
}
2021/12/24 08:49:28 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:49:33 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:49:33 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:49:35 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:49:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:49:40 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:49:45 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:49:45 Deploying app: TestDuplicateTimerFiringEventingRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:49:46 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn stored in temp store"
}
2021/12/24 08:49:51 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/24 08:49:56 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/24 08:50:01 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/24 08:50:06 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get deployed
2021/12/24 08:50:06 App: TestDuplicateTimerFiringEventingRebIn got deployed
2021/12/24 08:50:06 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:50:06 Starting bucket ops to source bucket
2021/12/24 08:50:06 Removing node(s): [] from the cluster
2021/12/24 08:50:11 Rebalance progress: 12.5
2021/12/24 08:50:16 Rebalance progress: 12.5
2021/12/24 08:50:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:50:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:50:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:50:22 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:50:27 Rebalance progress: 12.5
2021/12/24 08:50:32 Rebalance progress: 12.5
2021/12/24 08:50:37 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:50:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:50:37 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:50:37 Waiting for src bucket item count to get to: 50000 curr count: 29508
2021/12/24 08:50:42 Waiting for src bucket item count to get to: 50000 curr count: 40896
2021/12/24 08:50:47 Waiting for src bucket item count to get to: 50000 curr count: 40896
2021/12/24 08:50:57 Waiting for dst bucket item count to get to: 50000 curr count: 34877
2021/12/24 08:51:02 Waiting for dst bucket item count to get to: 50000 curr count: 34877
2021/12/24 08:51:07 Waiting for dst bucket item count to get to: 50000 curr count: 41970
2021/12/24 08:51:12 Waiting for dst bucket item count to get to: 50000 curr count: 41970
2021/12/24 08:51:12 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:51:12 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:51:17 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:51:22 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:51:27 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:51:32 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:51:37 Waiting for app: TestDuplicateTimerFiringEventingRebIn to get un-deployed
2021/12/24 08:51:37 App: TestDuplicateTimerFiringEventingRebIn got un-deployed
2021/12/24 08:51:37 No eventing-con process running
2021/12/24 08:51:37 Function: TestDuplicateTimerFiringEventingRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebIn deleting in background"
}
2021/12/24 08:51:39 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:51:44 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:51:49 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:51:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:51:51 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:51:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:52:01 Waiting for dst bucket item count to get to: 0 curr count: 45396
2021/12/24 08:52:06 Waiting for dst bucket item count to get to: 0 curr count: 45396
2021/12/24 08:52:06 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebIn (169.85s)
=== RUN   TestDuplicateTimerFiringEventingRebOut
2021/12/24 08:52:11 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:52:11 Removing node(s): [] from the cluster
2021/12/24 08:52:16 Rebalance progress: 100
2021/12/24 08:52:21 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:52:21 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/24 08:52:21 No eventing-con process running
2021/12/24 08:52:21 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingRebOut not found"
}
2021/12/24 08:52:23 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:52:28 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:52:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:52:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:52:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:52:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:52:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:52:40 Deploying app: TestDuplicateTimerFiringEventingRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:52:41 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut stored in temp store"
}
2021/12/24 08:52:46 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/24 08:52:51 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/24 08:52:56 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/24 08:53:01 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get deployed
2021/12/24 08:53:01 App: TestDuplicateTimerFiringEventingRebOut got deployed
2021/12/24 08:53:01 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:53:01 Starting bucket ops to source bucket
2021/12/24 08:53:06 Rebalance progress: 12.5
2021/12/24 08:53:11 Rebalance progress: 12.5
2021/12/24 08:53:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:53:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:53:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:53:16 Waiting for src bucket item count to get to: 50000 curr count: 11170
2021/12/24 08:53:21 Waiting for src bucket item count to get to: 50000 curr count: 25232
2021/12/24 08:53:26 Waiting for src bucket item count to get to: 50000 curr count: 25232
2021/12/24 08:53:31 Waiting for src bucket item count to get to: 50000 curr count: 36008
2021/12/24 08:53:36 Waiting for src bucket item count to get to: 50000 curr count: 36008
2021/12/24 08:53:41 Waiting for src bucket item count to get to: 50000 curr count: 46055
2021/12/24 08:53:46 Waiting for src bucket item count to get to: 50000 curr count: 46055
2021/12/24 08:53:56 Waiting for dst bucket item count to get to: 50000 curr count: 35814
2021/12/24 08:54:01 Waiting for dst bucket item count to get to: 50000 curr count: 35814
2021/12/24 08:54:06 Waiting for dst bucket item count to get to: 50000 curr count: 35814
2021/12/24 08:54:06 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:54:06 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:54:11 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:54:16 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:54:22 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:54:27 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:54:32 Waiting for app: TestDuplicateTimerFiringEventingRebOut to get un-deployed
2021/12/24 08:54:32 App: TestDuplicateTimerFiringEventingRebOut got un-deployed
2021/12/24 08:54:32 No eventing-con process running
2021/12/24 08:54:32 Function: TestDuplicateTimerFiringEventingRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingRebOut deleting in background"
}
2021/12/24 08:54:34 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:54:39 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:54:44 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:54:49 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:54:49 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:54:50 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:54:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:55:00 Waiting for dst bucket item count to get to: 0 curr count: 20641
2021/12/24 08:55:05 Waiting for dst bucket item count to get to: 0 curr count: 20641
2021/12/24 08:55:05 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingRebOut (179.43s)
=== RUN   TestDuplicateTimerFiringEventingSwapReb
2021/12/24 08:55:15 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:55:15 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/24 08:55:15 No eventing-con process running
2021/12/24 08:55:15 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringEventingSwapReb not found"
}
2021/12/24 08:55:17 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:55:22 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:55:22 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:55:24 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:55:29 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:55:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:55:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:55:34 Deploying app: TestDuplicateTimerFiringEventingSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:55:35 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb stored in temp store"
}
2021/12/24 08:55:40 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/24 08:55:45 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/24 08:55:50 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/24 08:55:55 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get deployed
2021/12/24 08:55:55 App: TestDuplicateTimerFiringEventingSwapReb got deployed
2021/12/24 08:55:55 Adding node: https://127.0.0.1:19001 with role: eventing to the cluster
2021/12/24 08:55:55 Starting bucket ops to source bucket
2021/12/24 08:55:55 Removing node(s): [] from the cluster
2021/12/24 08:56:00 Rebalance progress: 12.5
2021/12/24 08:56:05 Rebalance progress: 12.5
2021/12/24 08:56:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:56:10 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 08:56:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:56:16 Rebalance progress: 12.5
2021/12/24 08:56:21 Rebalance progress: 12.5
2021/12/24 08:56:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:56:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 08:56:32 Rebalance progress: 12.5
2021/12/24 08:56:37 Rebalance progress: 12.5
2021/12/24 08:56:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:56:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:56:47 Waiting for dst bucket item count to get to: 50000 curr count: 34056
2021/12/24 08:56:52 Waiting for dst bucket item count to get to: 50000 curr count: 34056
2021/12/24 08:56:57 Waiting for dst bucket item count to get to: 50000 curr count: 40276
2021/12/24 08:57:02 Waiting for dst bucket item count to get to: 50000 curr count: 40276
2021/12/24 08:57:02 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:57:02 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:57:07 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:57:12 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:57:17 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:57:22 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:57:27 Waiting for app: TestDuplicateTimerFiringEventingSwapReb to get un-deployed
2021/12/24 08:57:27 App: TestDuplicateTimerFiringEventingSwapReb got un-deployed
2021/12/24 08:57:27 No eventing-con process running
2021/12/24 08:57:27 Function: TestDuplicateTimerFiringEventingSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringEventingSwapReb deleting in background"
}
2021/12/24 08:57:29 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:57:34 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:57:39 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 08:57:39 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:57:41 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:57:46 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:57:51 Waiting for dst bucket item count to get to: 0 curr count: 39271
2021/12/24 08:57:56 Waiting for dst bucket item count to get to: 0 curr count: 39271
2021/12/24 08:57:56 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringEventingSwapReb (170.42s)
=== RUN   TestDuplicateTimerFiringKVRebIn
2021/12/24 08:58:06 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 08:58:06 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/24 08:58:06 No eventing-con process running
2021/12/24 08:58:06 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebIn not found"
}
2021/12/24 08:58:07 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:58:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:58:12 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 08:58:14 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:58:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:58:19 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:58:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 08:58:24 Deploying app: TestDuplicateTimerFiringKVRebIn. Item count src bucket: 0 dst bucket: 0
2021/12/24 08:58:25 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn stored in temp store"
}
2021/12/24 08:58:30 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/24 08:58:35 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/24 08:58:40 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/24 08:58:45 Waiting for app: TestDuplicateTimerFiringKVRebIn to get deployed
2021/12/24 08:58:45 App: TestDuplicateTimerFiringKVRebIn got deployed
2021/12/24 08:58:45 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 08:58:45 Starting bucket ops to source bucket
2021/12/24 08:58:46 Removing node(s): [] from the cluster
2021/12/24 08:58:51 Rebalance progress: 12.5
2021/12/24 08:58:56 Rebalance progress: 12.5
2021/12/24 08:59:01 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:59:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:59:01 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:59:01 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 08:59:06 Rebalance progress: 12.5
2021/12/24 08:59:11 Rebalance progress: 12.5
2021/12/24 08:59:16 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:59:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 08:59:16 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 08:59:16 Waiting for src bucket item count to get to: 50000 curr count: 26711
2021/12/24 08:59:21 Waiting for src bucket item count to get to: 50000 curr count: 39182
2021/12/24 08:59:26 Waiting for src bucket item count to get to: 50000 curr count: 39182
2021/12/24 08:59:31 Waiting for src bucket item count to get to: 50000 curr count: 49519
2021/12/24 08:59:36 Waiting for src bucket item count to get to: 50000 curr count: 49519
2021/12/24 08:59:46 Waiting for dst bucket item count to get to: 50000 curr count: 47545
2021/12/24 08:59:51 Waiting for dst bucket item count to get to: 50000 curr count: 47545
2021/12/24 08:59:51 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 08:59:51 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 08:59:56 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 09:00:01 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 09:00:07 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 09:00:12 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 09:00:17 Waiting for app: TestDuplicateTimerFiringKVRebIn to get un-deployed
2021/12/24 09:00:17 App: TestDuplicateTimerFiringKVRebIn got un-deployed
2021/12/24 09:00:17 No eventing-con process running
2021/12/24 09:00:17 Function: TestDuplicateTimerFiringKVRebIn delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebIn deleting in background"
}
2021/12/24 09:00:19 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:00:24 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:00:29 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 09:00:29 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:00:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:00:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:00:40 Waiting for dst bucket item count to get to: 0 curr count: 21912
2021/12/24 09:00:45 Waiting for dst bucket item count to get to: 0 curr count: 21912
2021/12/24 09:00:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebIn (169.60s)
=== RUN   TestDuplicateTimerFiringKVRebOut
2021/12/24 09:00:50 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:00:51 Removing node(s): [] from the cluster
2021/12/24 09:00:56 Rebalance progress: 100
2021/12/24 09:01:01 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:01:01 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/24 09:01:01 No eventing-con process running
2021/12/24 09:01:01 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVRebOut not found"
}
2021/12/24 09:01:02 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:01:07 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:01:07 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:01:09 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:01:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:01:14 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:01:19 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:01:19 Deploying app: TestDuplicateTimerFiringKVRebOut. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:01:20 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut stored in temp store"
}
2021/12/24 09:01:25 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/24 09:01:30 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/24 09:01:35 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/24 09:01:40 Waiting for app: TestDuplicateTimerFiringKVRebOut to get deployed
2021/12/24 09:01:41 App: TestDuplicateTimerFiringKVRebOut got deployed
2021/12/24 09:01:41 Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:01:41 Starting bucket ops to source bucket
2021/12/24 09:01:46 Rebalance progress: 12.5
2021/12/24 09:01:51 Rebalance progress: 12.5
2021/12/24 09:01:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:01:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:01:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:01:56 Waiting for src bucket item count to get to: 50000 curr count: 11837
2021/12/24 09:02:01 Waiting for src bucket item count to get to: 50000 curr count: 23680
2021/12/24 09:02:06 Waiting for src bucket item count to get to: 50000 curr count: 23680
2021/12/24 09:02:11 Waiting for src bucket item count to get to: 50000 curr count: 34049
2021/12/24 09:02:16 Waiting for src bucket item count to get to: 50000 curr count: 34049
2021/12/24 09:02:21 Waiting for src bucket item count to get to: 50000 curr count: 45729
2021/12/24 09:02:26 Waiting for src bucket item count to get to: 50000 curr count: 45729
2021/12/24 09:02:31 Waiting for src bucket item count to get to: 50000 curr count: 45729
2021/12/24 09:02:41 Waiting for dst bucket item count to get to: 50000 curr count: 42951
2021/12/24 09:02:46 Waiting for dst bucket item count to get to: 50000 curr count: 42951
2021/12/24 09:02:46 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 09:02:46 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 09:02:51 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:02:56 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:03:01 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:03:06 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:03:11 Waiting for app: TestDuplicateTimerFiringKVRebOut to get un-deployed
2021/12/24 09:03:11 App: TestDuplicateTimerFiringKVRebOut got un-deployed
2021/12/24 09:03:11 No eventing-con process running
2021/12/24 09:03:12 Function: TestDuplicateTimerFiringKVRebOut delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVRebOut deleting in background"
}
2021/12/24 09:03:13 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:03:18 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:03:23 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 09:03:28 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 09:03:28 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:03:30 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:03:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:03:40 Waiting for dst bucket item count to get to: 0 curr count: 6302
2021/12/24 09:03:45 Waiting for dst bucket item count to get to: 0 curr count: 6302
2021/12/24 09:03:45 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVRebOut (179.73s)
=== RUN   TestDuplicateTimerFiringKVSwapReb
2021/12/24 09:03:55 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:03:55 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/24 09:03:55 No eventing-con process running
2021/12/24 09:03:55 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: TestDuplicateTimerFiringKVSwapReb not found"
}
2021/12/24 09:03:57 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:04:02 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:04:02 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:04:04 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:04:09 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:04:09 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:04:14 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:04:14 Deploying app: TestDuplicateTimerFiringKVSwapReb. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:04:15 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb stored in temp store"
}
2021/12/24 09:04:20 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/24 09:04:25 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/24 09:04:30 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/24 09:04:35 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get deployed
2021/12/24 09:04:35 App: TestDuplicateTimerFiringKVSwapReb got deployed
2021/12/24 09:04:35 Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:04:35 Starting bucket ops to source bucket
2021/12/24 09:04:35 Removing node(s): [] from the cluster
2021/12/24 09:04:40 Rebalance progress: 12.5
2021/12/24 09:04:45 Rebalance progress: 12.5
2021/12/24 09:04:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:04:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:04:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:04:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:04:51 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:04:56 Rebalance progress: 12.5
2021/12/24 09:05:01 Rebalance progress: 12.5
2021/12/24 09:05:06 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:05:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:05:06 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:05:06 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:05:11 Rebalance progress: 12.5
2021/12/24 09:05:16 Rebalance progress: 12.5
2021/12/24 09:05:21 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:05:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:05:21 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:05:21 Waiting for src bucket item count to get to: 50000 curr count: 49411
2021/12/24 09:05:26 Waiting for src bucket item count to get to: 50000 curr count: 49411
2021/12/24 09:05:36 Waiting for dst bucket item count to get to: 50000 curr count: 46869
2021/12/24 09:05:41 Waiting for dst bucket item count to get to: 50000 curr count: 46869
2021/12/24 09:05:41 src & dst bucket item count matched up. src bucket count: 50000 dst bucket count: 50000
2021/12/24 09:05:41 Post SET ops. Expected item count: 50000 got 50000
2021/12/24 09:05:46 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:05:51 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:05:56 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:06:01 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:06:06 Waiting for app: TestDuplicateTimerFiringKVSwapReb to get un-deployed
2021/12/24 09:06:06 App: TestDuplicateTimerFiringKVSwapReb got un-deployed
2021/12/24 09:06:06 No eventing-con process running
2021/12/24 09:06:06 Function: TestDuplicateTimerFiringKVSwapReb delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: TestDuplicateTimerFiringKVSwapReb deleting in background"
}
2021/12/24 09:06:08 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:06:13 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:06:18 Waiting for dst bucket item count to get to: 0 curr count: 50000
2021/12/24 09:06:18 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:06:20 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:06:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:06:30 Waiting for dst bucket item count to get to: 0 curr count: 3844
2021/12/24 09:06:35 Waiting for dst bucket item count to get to: 0 curr count: 3844
2021/12/24 09:06:35 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestDuplicateTimerFiringKVSwapReb (169.92s)
=== RUN   TestKVRebOnUpdateBucketOpOneByOne
2021/12/24 09:06:45 Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:06:45 App: bucket_op_on_update got un-deployed
2021/12/24 09:06:45 No eventing-con process running
2021/12/24 09:06:45 Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:06:47 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:06:52 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:06:52 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:06:54 Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:06:59 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:06:59 src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:04 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:07:04 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:07:05 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:07:10 Starting bucket ops to source bucket
2021/12/24 09:07:15 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:07:20 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:07:25 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:07:25 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:07:25 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:07:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:07:30 Rebalance progress: 12.5
2021/12/24 09:07:35 Rebalance progress: 12.5
2021/12/24 09:07:40 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:40 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:07:40 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:07:40 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:07:45 Rebalance progress: 12.5
2021/12/24 09:07:50 Rebalance progress: 12.5
2021/12/24 09:07:55 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:07:55 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:07:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/24 09:07:56 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:08:01 Rebalance progress: 12.5
2021/12/24 09:08:06 Rebalance progress: 12.5
2021/12/24 09:08:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:08:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:08:16 Rebalance progress: 12.5
2021/12/24 09:08:21 Rebalance progress: 12.5
2021/12/24 09:08:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:08:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:08:31 Rebalance progress: 12.5
2021/12/24 09:08:36 Rebalance progress: 12.5
2021/12/24 09:08:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:08:41 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 09:08:46 Rebalance progress: 12.5
2021/12/24 09:08:51 Rebalance progress: 12.5
2021/12/24 09:08:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:08:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:09:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:09:06 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:09:12 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:09:17 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:09:22 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:09:22 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:09:22 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:09:22 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:09:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:09:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:09:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10651
2021/12/24 09:09:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 10651
2021/12/24 09:09:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:09:40 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:09:45 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:09:50 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5134
2021/12/24 09:09:55 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5134
2021/12/24 09:09:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpOneByOne (200.42s)
=== RUN   TestKVForceFailover
2021/12/24 09:10:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:10:10 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:10:10 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:10:10 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:10:12 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:10:17 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:10:17 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:10:19 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:10:24 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:10:24 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:10:24 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:10:25 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:10:30 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:10:30 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:10:30 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:10:31 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:10:31 Starting bucket ops to source bucket
2021/12/24 09:10:36 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:10:41 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:10:46 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:10:51 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:10:51 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:10:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:10:51 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:10:51 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/24 09:11:11 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/24 09:11:11 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: eventing to the cluster
2021/12/24 09:11:16 Rebalance progress: 12.5
2021/12/24 09:11:21 Rebalance progress: 12.5
2021/12/24 09:11:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:11:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:11:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:11:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002] from the cluster
2021/12/24 09:11:31 Rebalance progress: 12.5
2021/12/24 09:11:36 Rebalance progress: 12.5
2021/12/24 09:11:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:11:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:11:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:11:47 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:11:52 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:11:57 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:12:02 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:12:07 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:12:07 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:12:07 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:12:07 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:12:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:12:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:12:19 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7051
2021/12/24 09:12:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:12:21 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:12:26 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:12:31 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5413
2021/12/24 09:12:36 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 5413
2021/12/24 09:12:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVForceFailover (160.34s)
=== RUN   TestKVHardFailoverRecovery
2021/12/24 09:12:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:12:46 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:12:46 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:12:46 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:12:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:12:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:12:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:12:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:12:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:12:59 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:13:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:13:04 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:13:05 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:13:10 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:13:15 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:13:20 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:13:25 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:13:25 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:13:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:13:25 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:13:25 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:13:26 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:13:31 Rebalance progress: 12.5
2021/12/24 09:13:36 Rebalance progress: 12.5
2021/12/24 09:13:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:13:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:13:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:13:41 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
2021/12/24 09:14:41 clusterSetup.go:237: Kicking off failover recovery, type: full
2021/12/24 09:14:46 Rebalance progress: 12.5
2021/12/24 09:14:51 Rebalance progress: 12.5
2021/12/24 09:14:56 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:14:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:14:56 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:14:56 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:15:01 Rebalance progress: 12.5
2021/12/24 09:15:06 Rebalance progress: 12.5
2021/12/24 09:15:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:15:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:15:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:15:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:15:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:15:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:15:32 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:15:37 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:15:37 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:15:37 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:15:37 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:15:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:15:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:15:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12056
2021/12/24 09:15:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:15:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:15:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:16:00 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3719
2021/12/24 09:16:05 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 3719
2021/12/24 09:16:05 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVHardFailoverRecovery (209.49s)
=== RUN   TestKVRebOnUpdateBucketOpAllAtOnce
2021/12/24 09:16:15 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:16:15 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:16:15 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:16:15 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:16:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:16:22 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:16:22 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:16:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:16:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:16:29 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:16:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:16:34 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:16:35 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:16:40 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:16:45 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:16:50 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:16:55 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:16:55 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:16:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:16:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:16:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:16:55 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:16:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/24 09:16:56 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:17:01 Rebalance progress: 12.5
2021/12/24 09:17:06 Rebalance progress: 12.5
2021/12/24 09:17:11 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:17:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:17:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:17:11 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001 127.0.0.1:19002 127.0.0.1:19003] from the cluster
2021/12/24 09:17:16 Rebalance progress: 12.5
2021/12/24 09:17:21 Rebalance progress: 12.5
2021/12/24 09:17:26 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:17:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:17:26 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:17:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:17:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:17:42 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:17:47 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:17:52 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:17:52 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:17:52 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:17:52 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:17:54 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:17:59 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:18:04 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4627
2021/12/24 09:18:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4627
2021/12/24 09:18:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:18:10 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:18:15 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:18:20 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1981
2021/12/24 09:18:25 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1981
2021/12/24 09:18:25 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebOnUpdateBucketOpAllAtOnce (140.16s)
=== RUN   TestKVSwapRebOnUpdateBucketOp
2021/12/24 09:18:35 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:18:35 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:18:35 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:18:35 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:18:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:18:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:18:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:18:44 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:18:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:18:49 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:18:49 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:18:49 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:18:50 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:18:55 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:19:00 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:19:05 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:19:10 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:19:10 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:10 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:19:10 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:19:10 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:19:15 Rebalance progress: 12.5
2021/12/24 09:19:20 Rebalance progress: 12.5
2021/12/24 09:19:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:19:25 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:19:26 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:19:31 Rebalance progress: 12.5
2021/12/24 09:19:36 Rebalance progress: 12.5
2021/12/24 09:19:41 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:41 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:19:41 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:19:47 Rebalance progress: 12.5
2021/12/24 09:19:52 Rebalance progress: 12.5
2021/12/24 09:19:57 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:19:57 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:20:02 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:20:07 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:20:12 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:20:17 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:20:22 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:20:22 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:20:22 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:20:22 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:20:24 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:20:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:20:34 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6138
2021/12/24 09:20:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 6138
2021/12/24 09:20:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:20:41 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:20:46 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:20:51 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4837
2021/12/24 09:20:56 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 4837
2021/12/24 09:20:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:21:01 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:21:01 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:21:01 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:21:01 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:21:03 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:08 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:21:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:14 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebOnUpdateBucketOp (168.94s)
=== RUN   TestKVRebalanceWithMultipleHandlers
2021/12/24 09:21:24 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:21:24 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:21:24 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:21:24 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:21:26 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:31 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:31 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:21:33 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:38 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:38 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:21:43 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:21:43 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/24 09:21:43 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:21:43 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/24 09:21:45 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:50 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:50 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:21:51 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:56 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:56 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:21:57 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:21:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:21:57 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:21:59 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/24 09:22:04 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:22:09 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:22:14 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:22:19 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:22:19 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/24 09:22:24 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:22:29 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:22:34 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:22:34 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:22:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:22:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:22:34 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:22:34 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:22:39 Rebalance progress: 12.5
2021/12/24 09:22:44 Rebalance progress: 12.5
2021/12/24 09:22:49 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:22:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:22:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:22:49 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:22:50 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:22:55 Rebalance progress: 12.5
2021/12/24 09:23:00 Rebalance progress: 12.5
2021/12/24 09:23:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:23:05 clusterSetup.go:245: Adding node: https://127.0.0.1:19003 with role: kv to the cluster
2021/12/24 09:23:05 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:23:10 Rebalance progress: 12.5
2021/12/24 09:23:15 Rebalance progress: 12.5
2021/12/24 09:23:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:23:20 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:23:25 Rebalance progress: 12.5
2021/12/24 09:23:30 Rebalance progress: 12.5
2021/12/24 09:23:35 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:35 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:23:35 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:23:40 Rebalance progress: 12.5
2021/12/24 09:23:45 Rebalance progress: 12.5
2021/12/24 09:23:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:23:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:23:50 clusterSetup.go:253: Removing node(s): [127.0.0.1:19003] from the cluster
2021/12/24 09:23:55 Rebalance progress: 12.5
2021/12/24 09:24:00 Rebalance progress: 12.5
2021/12/24 09:24:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:24:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:24:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:24:10 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:24:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:24:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:24:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:24:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:24:31 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:24:31 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:24:31 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:24:36 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:24:41 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:24:46 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12162
2021/12/24 09:24:46 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:24:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:24:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:24:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 12162
2021/12/24 09:24:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:25:02 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:25:08 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:25:13 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:25:18 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:25:23 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:25:23 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/24 09:25:23 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:25:23 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/24 09:25:25 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:25:30 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:25:30 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:25:32 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:25:37 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:25:37 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebalanceWithMultipleHandlers (262.37s)
=== RUN   TestKVSwapRebalanceWithMultipleHandlers
2021/12/24 09:25:47 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:25:47 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:25:47 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:25:47 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:25:48 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:25:53 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:25:53 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:25:55 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:00 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:26:00 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:26:05 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:26:05 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/24 09:26:05 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:26:05 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: sys_test_bucket_op not found"
}
2021/12/24 09:26:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:26:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:26:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:19 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:26:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:19 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:26:19 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:26:20 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:20 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:26:20 eventingSetup.go:125: Deploying app: sys_test_bucket_op. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:26:21 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op stored in temp store"
}
2021/12/24 09:26:26 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:26:31 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:26:36 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:26:41 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:26:41 clusterSetup.go:354: App: bucket_op_on_update got deployed
2021/12/24 09:26:46 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:26:51 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:26:56 clusterSetup.go:346: Waiting for app: sys_test_bucket_op to get deployed
2021/12/24 09:26:56 clusterSetup.go:354: App: sys_test_bucket_op got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:26:56 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:26:56 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:26:57 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:27:02 Rebalance progress: 12.5
2021/12/24 09:27:07 Rebalance progress: 12.5
2021/12/24 09:27:12 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:12 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:27:12 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:27:12 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:27:17 Rebalance progress: 12.5
2021/12/24 09:27:22 Rebalance progress: 12.5
2021/12/24 09:27:27 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:27 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:27:27 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:27:32 Rebalance progress: 12.5
2021/12/24 09:27:37 Rebalance progress: 12.5
2021/12/24 09:27:42 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:27:42 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:27:48 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:27:53 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:27:58 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:28:03 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:28:08 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:28:13 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:28:13 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:28:13 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:28:13 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:28:17 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:28:22 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:28:27 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7639
2021/12/24 09:28:27 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:28:29 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:28:34 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:28:39 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 7639
2021/12/24 09:28:39 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:28:44 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:28:49 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:28:55 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:29:00 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:29:05 clusterSetup.go:397: Waiting for app: sys_test_bucket_op to get un-deployed
2021/12/24 09:29:05 clusterSetup.go:405: App: sys_test_bucket_op got un-deployed
2021/12/24 09:29:05 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:29:05 eventingSetup.go:418: Function: sys_test_bucket_op delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: sys_test_bucket_op deleting in background"
}
2021/12/24 09:29:07 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:29:12 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:29:12 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:29:14 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:29:19 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:29:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVSwapRebalanceWithMultipleHandlers (221.90s)
=== RUN   TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne
2021/12/24 09:29:29 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:29:29 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:29:29 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:29:29 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:29:30 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:29:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:29:35 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:29:37 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:29:42 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:29:42 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:29:47 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:29:47 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:29:48 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:29:53 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:29:58 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:30:03 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:30:08 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:30:08 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:08 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:30:08 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:30:09 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:30:29 clusterSetup.go:165: Stopping rebalance
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:29 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:39 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:30:39 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:30:44 Rebalance progress: 12.5
2021/12/24 09:30:49 Rebalance progress: 12.5
2021/12/24 09:30:54 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:30:54 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:30:54 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:30:55 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:31:00 Rebalance progress: 12.5
2021/12/24 09:31:05 Rebalance progress: 12.5
2021/12/24 09:31:10 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:31:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:31:10 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:31:10 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:31:15 Rebalance progress: 12.5
2021/12/24 09:31:20 Rebalance progress: 12.5
2021/12/24 09:31:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:31:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:31:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:31:30 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:31:35 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:31:40 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:31:45 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:31:51 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:31:51 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:31:51 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:31:51 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:31:52 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:31:57 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:32:02 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 9161
2021/12/24 09:32:07 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 9161
2021/12/24 09:32:07 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:32:09 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:32:14 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:32:19 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 9161
2021/12/24 09:32:19 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVRebStopStartKVOpsOnUpdateBucketOpOneByOne (180.64s)
=== RUN   TestKVFailoverOnUpdateBucketOp
2021/12/24 09:32:29 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:32:29 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:32:29 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:32:29 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 404 dump: {
 "name": "ERR_APP_NOT_FOUND_TS",
 "code": 13,
 "description": "Function: bucket_op_on_update not found"
}
2021/12/24 09:32:31 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:32:36 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:32:36 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:32:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:32:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:32:43 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:32:48 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:32:48 eventingSetup.go:125: Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:32:49 eventingSetup.go:102: Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:32:54 bucketOps.go:157: Starting bucket ops to source bucket
2021/12/24 09:32:59 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:33:04 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:33:09 clusterSetup.go:346: Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:33:09 clusterSetup.go:354: App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:09 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:33:09 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:33:09 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:33:15 Rebalance progress: 12.5
2021/12/24 09:33:20 Rebalance progress: 12.5
2021/12/24 09:33:25 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:25 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:33:25 clusterSetup.go:229: Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:35 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:33:35 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:33:40 Rebalance progress: 12.5
2021/12/24 09:33:45 Rebalance progress: 12.5
2021/12/24 09:33:50 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:33:50 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:33:50 clusterSetup.go:245: Adding node: https://127.0.0.1:19002 with role: kv to the cluster
2021/12/24 09:33:50 clusterSetup.go:253: Removing node(s): [127.0.0.1:19001] from the cluster
2021/12/24 09:33:55 Rebalance progress: 12.5
2021/12/24 09:34:00 Rebalance progress: 12.5
2021/12/24 09:34:05 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:34:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:34:05 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:34:05 clusterSetup.go:253: Removing node(s): [127.0.0.1:19002] from the cluster
2021/12/24 09:34:10 Rebalance progress: 12.5
2021/12/24 09:34:15 Rebalance progress: 12.5
2021/12/24 09:34:20 Rebalance progress: 100
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:34:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:34:20 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:34:25 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:34:30 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:34:35 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:34:41 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:34:46 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:34:46 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:34:46 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:34:46 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:34:47 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:34:52 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:34:57 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8600
2021/12/24 09:34:57 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:34:59 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:35:04 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:35:09 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 8600
2021/12/24 09:35:09 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestKVFailoverOnUpdateBucketOp (170.01s)
=== RUN   TestBootstrapAfterKVHardFailover
2021/12/24 09:35:14 clusterSetup.go:245: Adding node: https://127.0.0.1:19001 with role: kv to the cluster
2021/12/24 09:35:14 clusterSetup.go:253: Removing node(s): [] from the cluster
2021/12/24 09:35:19 Rebalance progress: 100
2021/12/24 09:35:19 Failing over: [127.0.0.1:19001]
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:35:49 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:35:50 Deploying app: bucket_op_on_update. Item count src bucket: 0 dst bucket: 0
2021/12/24 09:35:51 Context: Post to main store, response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update stored in temp store"
}
2021/12/24 09:35:56 Removing node(s): [] from the cluster
2021/12/24 09:35:56 Starting bucket ops to source bucket
2021/12/24 09:36:01 Rebalance failed. See logs for detailed reason. You can try again.
2021/12/24 09:36:06 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:36:11 Waiting for app: bucket_op_on_update to get deployed
2021/12/24 09:36:11 App: bucket_op_on_update got deployed
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:36:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

VB distribution across eventing nodes and their workers::

Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:36:11 waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 

DCP Stream statuses:

vbucket curr owner:


2021/12/24 09:36:16 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:36:21 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:36:26 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:36:31 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:36:36 clusterSetup.go:397: Waiting for app: bucket_op_on_update to get un-deployed
2021/12/24 09:36:36 clusterSetup.go:405: App: bucket_op_on_update got un-deployed
2021/12/24 09:36:36 eventingSetup.go:46: No eventing-con process running
2021/12/24 09:36:36 eventingSetup.go:418: Function: bucket_op_on_update delete from main store request response code: 200 dump: {
 "code": 0,
 "info": "Function: bucket_op_on_update deleting in background"
}
2021/12/24 09:36:38 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:36:43 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:36:48 eventingSetup.go:441: Waiting for dst bucket item count to get to: 0 curr count: 1508
2021/12/24 09:36:48 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
2021/12/24 09:36:50 eventingSetup.go:102: Context: Bucket flush, response code: 200 dump: 
Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused
2021/12/24 09:36:55 clusterSetup.go:81: waitForIndexes giving up: err Get "http://127.0.0.1:9108/getIndexStatus": dial tcp 127.0.0.1:9108: connect: connection refused response 
2021/12/24 09:36:55 eventingSetup.go:436: src & dst bucket item count matched up. src bucket count: 0 dst bucket count: 0
--- PASS: TestBootstrapAfterKVHardFailover (105.63s)
PASS
ok  	github.com/couchbase/eventing/tests/functional_tests	12030.586s

Rebalance tests took 3 hours, 20 minutes and 33 seconds.

Testing Succeeded


Version: versions-24.12.2021-02.00.cfg
Build Log: make-24.12.2021-02.00.log
Server Log: logs-24.12.2021-02.00.tar.gz

Finished