Skip to content

Not able to find the cause of failure of HDInsight Script action. #12

@sumanentc

Description

@sumanentc

Hi ,
i used custom script in ARM Template while creating HDInsight Spark Cluster. The template was getting executed successfully , but the script was not getting executed. I used some environment variables in my script which was not getting resolved. I spent a lot of time to figure it out the issue.
"scriptActions": [{
"name": "createLivyFolder",
"uri": "[parameters('scriptActionUri')]"
}]
The below challenges i faced

  1. Not able to get the path of the folder from where the script was executed in HDInsight Cluster.
  2. Not able to get the path of error log or Stack-trace of script action.

After spending a lot of time i got the log from the Azure blob storage and was able to get the cause of the problem.
Can you please add to the doc how to debug the script action in case of failure or get the stack trace in case of success/failure.

Please let us know if we want to use some environment variables in script action like $SPARK_HOME,
How to do that. The below statement was failing in script actions.
sudo cp postgresql-9.4.1210.jre7.jar $SPARK_HOME/jars/

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions