kodaroWriteTags function in SkySpark

JKotiadis · 2019-05-06 18:37:02 UTC

Hello, I am working on a project where we want to utilize the kodaroWriteTags function to be able to sync tags added via SkySpark back down to our N4 station. I have followed the documentation and had few issues up until this stage. below is a copy of the function I am using:

() => do
//Get the haystack connector specified
conn: read(haystackConn and clientTag)
grid: readAll(point)
//Remove columns only needed in SkySpark
grid = grid.removeCols([“haystackHis”, “haystackCur”, “haystackWrite”, “haystackWriteLevel”])
//Run the sync action for each entry in the grid provided
res: [].toGrid
grid.each pt =>
do
tags: pt.findAll v => v == marker()
//Run the action and get the grid result
resp: haystackInvokeAction(conn,pt->id,“syncTags”,tags)
resp.each r =>
do
res = res.addRow®
end
end
res
end

I tried this iteration after the standard one from the doc did not write back the tags I added in SkySpark. I added cmd, sp, or sensor to points depending on their type, and also, for testing purposes, I added total (a HS tag) to all of the points. Both this version and the standard one from the doc run on SkySpark and do not return any errors, but no tags are written when I check N4. I am not sure what I am doing wrong, any help would be much appreciated. Thank you!

kodaroJonathan · 2019-05-06 20:00:45 UTC

I can see that you have modified the example code slightly to sync only marker type tags but it would appear that the ID also needs to be present in the grid of the tags sent down to properly resolve the target. Why this shouldn’t be a requirement, I have confirmed this is the case currently so you will need to modify your code to add the point id back into the tags submitted.

You would not see any errors as this type of failure would only report itself in the meta data of the response message which you would not be seeing with this method. You could also capture the logging of this error by turning on the haystack device ops logging to warning in Niagara.

JKotiadis · 2019-05-07 14:57:29 UTC

I reverted the code back, so now I am simply running this:

writeTags

I am still not seeing the tags being written back down to the points in Niagara. What else can I do?Thanks.

kodaroJonathan · 2019-05-07 15:01:06 UTC

Can you provide a screen shot of the point in Niagara and it’s tags and also the same point in SkySpark to help verify this issue?

Also, what versions of SkySpark, Niagara and the haystack driver are you currently using?

It may also be helpful to run the haystackInvokeAction command with just a single point to see if you are receiving any type of errors that are hidden by the function attempting to sync all the points.

JKotiadis · 2019-05-07 16:08:31 UTC

Point_N4

SkySpark: 3.0.19
Niagara: 4.4.92.2
Haystack+: 1.0.19.2

I will test running the haystackInvokeAction on a single point. I have been having an issue with the connActor being busy and very long query times on the connector for points (less than 50). Querying other records is ok but points generally will timeout (despite me increasing the timeout window on both SS and N4 to 2 minutes), and it began around when I started testing the writeTags function.

Lastly, this may not be relevant but I am using the Digest authentication scheme instead of Basic HTTP (I know the documentation says otherwise). This was the only way I was able to get connected with my above combination of versions, is this ok?

JKotiadis · 2019-05-07 16:09:26 UTC

Point_SS

As a new user was limited to 1 image per post, here is the second one you requested.

kodaroJonathan · 2019-05-07 16:33:46 UTC

Digest instead of Basic is expected since Niagara 4.3 so that configuration is fine.

If you could run a single haystackInvokeAction with that point and show the full axon query that invokes it, that would be helfpul in determining if it is actually being accepted by Niagara. Similarly, if you turn on the dev ops logging in Niagara and monitor at the time the request is sent, it should lend some more info on what is failing to update the tags.

JKotiadis · 2019-05-10 12:34:39 UTC

I have done some additional testing and it appears that when I tried to write the tags it created another issue. I am able to query for the following and return the single result I would expect:

read(haystackConn).haystackCall(“read”,{filter:“hs:site”})

When I run a similar query but for equips, expecting 50 or so results, I get the following from SkySpark:

axon::EvalErr: Func failed: haystackCall(Obj conn,Str op,Obj req,Bool checked); args: (MapDict,Str,Dict1)
sys::IOErr: Bad HTTP response 500 Async Timeout [eval:1]
axon::FantomFn.callx (FantomFn.fan:132)
axon::Fn.callLazy (Fn.fan:71)
axon::Call.eval (Call.fan:39)
skyarcd::Context.evalOrReadAll (Context.fan:108)
skyarcd::SysLib.eval (SysLib.fan:92)
skyarcd::ApiWebMod.evalAllExpr (ApiMod.fan:284)
skyarcd::ApiWebMod.onEvalAll (ApiMod.fan:254)
fan.sys.List.each (List.java:602)
haystack::GbGrid.each (GridBuilder.fan:350)
skyarcd::ApiWebMod.onEvalAll (ApiMod.fan:250)
skyarcd::ApiWebMod.onService (ApiMod.fan:109)
skyarcd::HttpRootMod.onService (HttpMod.fan:213)
arcbeam::HttpChan.onReq (HttpChan.fan:97)
arcbeam::ChanActor.onReceivedReq (ChanActor.fan:130)
arcbeam::ChanActor.receive (ChanActor.fan:56)
concurrent::Actor._dispatch (Actor.java:234)
concurrent::Actor._work (Actor.java:204)
concurrent::ThreadPool$Worker.run (ThreadPool.java:262)
Cause:
sys::IOErr: Bad HTTP response 500 Async Timeout
concurrent::Future.get (Future.java:118)
connExt::ConnActor.sendSync (ConnActor.fan:120)
haystackExt::HaystackLib.toResult (HaystackLib.fan:350)
haystackExt::HaystackLib.haystackCall (HaystackLib.fan:59)
haystackExt::HaystackLib.haystackCall (HaystackLib.fan)
java.lang.reflect.Method.invoke (Unknown)
fan.sys.Method.invoke (Method.java:573)
fan.sys.Method$MethodFunc.callList (Method.java:212)
fan.sys.Method.callList (Method.java:138)
axon::FantomFn.doCall (FantomFn.fan:140)
axon::AxonContext.doCall (AxonContext.fan:145)
axon::FantomFn.callx (FantomFn.fan:128)
axon::Fn.callLazy (Fn.fan:71)
axon::Call.eval (Call.fan:39)
skyarcd::Context.evalOrReadAll (Context.fan:108)
skyarcd::SysLib.eval (SysLib.fan:92)
skyarcd::ApiWebMod.evalAllExpr (ApiMod.fan:284)
skyarcd::ApiWebMod.onEvalAll (ApiMod.fan:254)
fan.sys.List.each (List.java:602)
haystack::GbGrid.each (GridBuilder.fan:350)
skyarcd::ApiWebMod.onEvalAll (ApiMod.fan:250)
skyarcd::ApiWebMod.onService (ApiMod.fan:109)
skyarcd::HttpRootMod.onService (HttpMod.fan:213)
arcbeam::HttpChan.onReq (HttpChan.fan:97)
arcbeam::ChanActor.onReceivedReq (ChanActor.fan:130)
4 More…
sys::IOErr: Bad HTTP response 500 Async Timeout
skyarcd::Client.doCall (Client.fan:250)
skyarcd::Client.call (Client.fan:222)
haystackExt::HaystackConn.call (HaystackConn.fan:108)
haystackExt::HaystackConn.onCall (HaystackConn.fan:102)
haystackExt::HaystackConn.receive (HaystackConn.fan:37)
connExt::ConnActor.receive (ConnActor.fan:181)
concurrent::Actor._dispatch (Actor.java:234)
concurrent::Actor._work (Actor.java:204)
concurrent::ThreadPool$Worker.run (ThreadPool.java:262)

On Niagra I enabled the dev ops logging as you suggested and the following displays while the query is running:

org.projecthaystack.ParseException: Unexpected token: eof [line 1]
at org.projecthaystack.io.HZincReader.err(HZincReader.java:391)
at org.projecthaystack.io.HZincReader.err(HZincReader.java:390)
at org.projecthaystack.io.HZincReader.parseVal(HZincReader.java:172)
at org.projecthaystack.io.HZincReader.readVal(HZincReader.java:78)
at org.projecthaystack.io.HZincReader.readGrid(HZincReader.java:92)
at com.kodaro.haystack.device.ops.BHaystackReadOp.read(BHaystackReadOp.java:100)
at com.kodaro.haystack.device.ops.BHaystackReadOp.op(BHaystackReadOp.java:77)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.asyncOp(BHaystackDeviceOps.java:513)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.access$000(BHaystackDeviceOps.java:27)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps$HaystackWebOp.run(BHaystackDeviceOps.java:652)
at javax.baja.util.ThreadPoolWorker$WorkerThread.run(Unknown Source)
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.ResponseWriter.isOpen(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.flush(Unknown Source)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.respond(BHaystackDeviceOps.java:548)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.asyncOp(BHaystackDeviceOps.java:515)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.access$000(BHaystackDeviceOps.java:27)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps$HaystackWebOp.run(BHaystackDeviceOps.java:652)
at javax.baja.util.ThreadPoolWorker$WorkerThread.run(Unknown Source)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.HttpConnection$SendCallback.reset(Unknown Source)
at org.eclipse.jetty.server.HttpConnection$SendCallback.access$300(Unknown Source)
at org.eclipse.jetty.server.HttpConnection.send(Unknown Source)
at org.eclipse.jetty.server.HttpChannel.sendResponse(Unknown Source)
at org.eclipse.jetty.server.HttpChannel.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:167)
at org.eclipse.jetty.server.Utf8HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.respond(BHaystackDeviceOps.java:547)
… 4 more
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.ResponseWriter.isOpen(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.respond(BHaystackDeviceOps.java:547)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.asyncOp(BHaystackDeviceOps.java:520)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.access$000(BHaystackDeviceOps.java:27)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps$HaystackWebOp.run(BHaystackDeviceOps.java:652)
at javax.baja.util.ThreadPoolWorker$WorkerThread.run(Unknown Source)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.HttpConnection$SendCallback.reset(Unknown Source)
at org.eclipse.jetty.server.HttpConnection$SendCallback.access$300(Unknown Source)
at org.eclipse.jetty.server.HttpConnection.send(Unknown Source)
at org.eclipse.jetty.server.HttpChannel.sendResponse(Unknown Source)
at org.eclipse.jetty.server.HttpChannel.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at org.eclipse.jetty.server.HttpOutput.write(Unknown Source)
at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:167)
at org.eclipse.jetty.server.Utf8HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.HttpWriter.write(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at org.eclipse.jetty.server.ResponseWriter.write(Unknown Source)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.respond(BHaystackDeviceOps.java:547)
at com.kodaro.haystack.device.ops.BHaystackDeviceOps.asyncOp(BHaystackDeviceOps.java:515)

the axon used is as follows for the equip and point queries, respectively:

read(haystackConn).haystackCall(“read”,{filter:“hs:equip”})
read(haystackConn).haystackCall(“read”,{filter:“hs:point and rc:test”})

I added an actorTimeout tag to my connector and increased its window to 3 minutes but this did not help. I also increased the Op Timeout on the HaystackNetwork Local Device on the Niagara side, but to no avail. I was able to query for equips and points without issue prior to running the write tags function, please advise the best course of action. Thanks.

kodaroJonathan · 2019-05-10 14:48:33 UTC

To verify some timing on the Niagara side, I recommend running that NEQL query directly in Niagara to see if it is able to handle it in a timely fashion or if it fails completely. That will give some insight as to what Niagara is able to handle and if there is some underlying issue with the station failing to resolve that query.

JKotiadis · 2019-05-10 17:26:00 UTC

NEQL query directly in Niagara returns the expected results almost instantly. Both SkySpark and the Niagara Soft-JACE are each on separate Dell blade servers with 32 gigs of ram and 8 core CPU’s, so I am certain it isn’t a resource issue.

kodaroJonathan · 2019-05-10 17:39:23 UTC

Reviewing the stack trace again, there may be an encoding issue being sent through the axon call. Did the axon method for read get copy pasted from somewhere else or directly typed into folio? Sometimes copy pasting form other editor can introduce formatting that is not valid for zinc processing even though it’s not always detectable by just looking at it.

So I would recommend typing this entire command into folio without the use of copy paste to make sure there are no special characters or hidden formatting in the request.

read(haystackConn).haystackCall(“read”,{filter:“hs:site”})

I also noticed in previous posts you had the haystack connector being resolved differently in the write tags method.

read(haystackConn and rockCenter)

Are there multiple haystack connectors in the project that could be causing conflicts without this additional tag?

JKotiadis · 2019-05-10 17:52:08 UTC

I typed the command initially for site and then modified it for various other queries like equip and point. I just retyped it and site queries, but equip and point timeout.

That connector is the only connector present, I could just filter for:

read(haystackConn)

I am just in the habit of identifying connectors via a marker tag as more may be added later.

kodaroJonathan · 2019-05-10 18:04:44 UTC

So site is not timing out but equip and point are? Are these also fast queries directly in NEQL in niagara? If so, what is the current timeout on the local device web ops? SkySpark actor is still 3 min I presume.

JKotiadis · 2019-05-10 18:38:28 UTC

Correct, Site returns the expected results, and equip and point timeout. I posted the exact query in my previous post. They are NEQL queries within search, something like hs:point and rc:test, where rc:test is a custom marker tag. The local device web op timeout is 2 minutes, and SkySpark is currently 3 min, correct.

kodaroJonathan · 2019-05-10 19:38:26 UTC

Just to back track a few steps, you say this read error occurred after the tag write updates. Were you able to confirm the tags you were attempting to write to a single point took effect? Did the write that happened possibly have references as a part of them that could have created new relationships in Niagara? The read process of the driver does attempt to resolve parent relationships and if there is a circular loop of relations, then this could cause a timeout issue.

You may also want to try a neql query that would return a single point to make sure there is not some other type of encoding issues going on. You could slowly expand out the number of results after that to determine whether or not this is indeed some issue with the driver, tags or possible network slowness between the two systems causing the timeouts and/or other errors.