RAKF Problem

[juergen.winkelmann]

Hi All,

I recently installed the TK3 system (the original one published on
Volker Bandtle's website, not the shadow update) under Hercules/390. Now
I'm trying to use RAKF on that system and experience a weird problem:

    * Submitting jobs from TSO with user and password coded on the
jobcard works as expected as long as the job doesn't contain DD
statements that allocate new dataset (i.e. contains only DD statements
with DISP=SHR or DISP=OLD)
    * As soon as a DISP=(,KEEP) or DISP=(,CATLG) is requested one of the
following happens:

    1. The job runs normal... which is fine
    2. The job terminates with the following messages:
18.10.28 JOB    6  $HASP373 JUERGENG STARTED - INIT  1 - CLASS A - SYS
MVS1
18.10.28 JOB    6  IEF403I JUERGENG - STARTED - TIME=18.10.28
18.10.28 JOB    6  IEC999I IGG0325A,INIT
18.10.28 JOB    6  IEA911E COMPLETE DUMP ON SYS1.DUMP00
                    FOR ASID (0009)
                     ERROR ID = SEQ00025 CPU00 ASID0009 TIME18.10.27.7
18.10.28 JOB    6  IEFACTRT - Stepname  Procstep  Program   Retcode
18.10.28 JOB    6  JUERGENG   GENER               IEBGENER  *FLUSH*
18.10.28 JOB    6  IEF453I JUERGENG - JOB FAILED - JCL ERROR -
TIME=18.10.28
18.10.28 JOB    6  $HASP395 JUERGENG ENDED
     1     //JUERGENG JOB  (SETUP),
           //             'Copy Files      ',
           //             CLASS=A,
           //             MSGCLASS=X,
           //             MSGLEVEL=(1,1),
           //             NOTIFY=JUERGEN,
           //             USER=JUERGEN,PASSWORD=
     2     //GENER   EXEC PGM=IEBGENER
     3     //SYSUT1   DD  DISP=SHR,DSN=JUERGEN.TEST.BMP(ARNISEE)
     4     //SYSUT2   DD  UNIT=SYSDA,DISP=(,CATLG),SPACE=(CYL,(1,1)),
           //             DSN=JUERGEN.ARNISEE.BMP
     5     //SYSPRINT DD  SYSOUT=*
     6     //SYSIN    DD  DUMMY
IEC999I IGG0325A,INIT
IEF197I SYSTEM ERROR DURING ALLOCATION

Once case 2. occured, the initiator abend will reoccur everytime a job
with a NEW allocation request is executed by the same intiator, until
the initiator is stopped and restarted. Then case 1. occurs exactly one
time and all following jobs suffer again the case 2. abends... and so
on. So, in other words, it is possible to run exactly one job with a NEW
allocation on an initiator, then the initiator needs to be stopped and
started, before it will be able to run another job of that kind.

I found lots of messages in this group dated around August 2005
discussing very similar problems and in one message Phil Dickinson
mentions he uploaded a changed version of RAKF with an eyeball of
23-08-05. The version I downloaded from the files section of this group
and installed on my system has an eyeball of 08-08-05 and thus seems to
be an older version. After this message, the problem seems gone... or at
least discussions stopped.

So my questions are: Does anybody know about this old problem and how to
solve it? Is somewhere a newer vorsion than 08-08-05 of RAKF available?

Thanks a lot in advance for any comments!

Cheers, Juergen

[halfmeg]

> juergen.winkelmann wrote:

><snip>

> So my questions are:

> Does anybody know about this old problem and how to solve it?

Not familiar with the specific problem as relates to RAKF.  Original TK3 that you are running did have several DASD images which may cause problems when writing a dataset.

> Is somewhere a newer vorsion than 08-08-05 of RAKF available?

RAKF.AWS in the files section is what you are looking for.  Not all of the members were reassembled.  Check RAKFINIT for the later assembly date.

Your post has had another result though.  The TK3 update currently being worked on had a bug in the parmlib, now fixed.

Phil

[juergen.winkelmann]

Hi Phil, thanks for your reply!

--- In H390-MVS@..., "halfmeg" <opplr@...> wrote:

>
> > juergen.winkelmann wrote:
>
> ><snip>
>
> > So my questions are:
>
> > Does anybody know about this old problem and how to solve it?
>
> Not familiar with the specific problem as relates to RAKF.  Original TK3 that you are running did have several DASD images which may cause problems when writing a dataset.
>

As the problem doesn't occur when RAKF isn't active, I think, that a potential disk write problem can be ruled out.
>
> > Is somewhere a newer vorsion than 08-08-05 of RAKF available?
>
> RAKF.AWS in the files section is what you are looking for.  Not all of the members were reassembled.  Check RAKFINIT for the later assembly date.
>
Yes, you're right. I did look only at the IGC0013x modules. RAKFINIT has the correct eyeball, so I already have the latest version. I still don't have a clue what the cause of that weird problem might be... except from this, RAKF works flawlessly.
>
> Your post has had another result though.  The TK3 update currently being worked on had a bug in the parmlib, now fixed.
>
good point... btw.: did anyone notice that the IEASYS00 parmlib member in TK3 has a bug? The comma after PAGE=(....,L), is missing, so the rest of the member is ignored at IPL. Even worse: This rest has a syntax error, it reads at the end WTOEPLY=10 instead of WTORPLY=10, which then leads to complaints at IPL as soon as one adds that missing comma after the PAGE parameter... just ignore if that's already fixed in the TK3 update ;-)
>
> Phil
>

[Gerhard Postpischil]

juergen.winkelmann wrote:
> good point... btw.: did anyone notice that the IEASYS00
> parmlib member in TK3 has a bug? The comma after
> PAGE=(....,L), is missing, so the rest of the member is
> ignored at IPL. Even worse: This rest has a syntax error, it
> reads at the end WTOEPLY=10 instead of WTORPLY=10, which then
> leads to complaints at IPL as soon as one adds that missing
> comma after the PAGE parameter... just ignore if that's
> already fixed in the TK3 update ;-)

Amazing - the error is in my tk3, and in two other members (I
made a backup copy, and a variation for testing). But somebody
caught the error, and it is fixed in SU1.

Gerhard Postpischil
Bradford, VT

[halfmeg]

> juergen.winkelmann wrote:

> > halfmeg wrote:

><snip>

> > Not familiar with the specific problem as relates to RAKF.
> >  Original TK3 that you are running did have several DASD images
> > which may cause problems when writing a dataset.

> As the problem doesn't occur when RAKF isn't active, I think, that a
> potential disk write problem can be ruled out.

I'm not sure about that as I had initiator blow-outs when writing a dataset some time back and wasn't using RAKF.

It would be good to pin this down as either a RAKF problem or other.  Your post didn't indicate an abend IIRC.  Would you briefly provide info as to exactly which items you modified in PARMLIB and post size, DCB info for the dataset along with the JCL ( IEBGENER ? ) so that it might be reproduced ?

><snip>

> good point... btw.: did anyone notice that the IEASYS00 parmlib
> member in TK3 has a bug?

><snip>

As Gerhard noted, the errors you mention have been corrected.

Phil

[juergen.winkelmann]

--- In H390-MVS@..., "halfmeg" <opplr@...> wrote:
>
> > juergen.winkelmann wrote:
>
> > As the problem doesn't occur when RAKF isn't active, I think, that a
> > potential disk write problem can be ruled out.
>
> I'm not sure about that as I had initiator blow-outs when writing a
dataset some time back and wasn't using RAKF.
>
> It would be good to pin this down as either a RAKF problem or other.
Your post didn't indicate an abend IIRC. Would you briefly provide info
as to exactly which items you modified in PARMLIB and post size, DCB
info for the dataset along with the JCL ( IEBGENER ? ) so that it might
be reproduced ?

Hi Phil,

thanks for your reply and sorry for my late follow up... I was three
weeks in holidays (Yosemite proved to be more fascinating than MVS  [;)]
)

I'm still convinced, that it's a RAKF problem: I now ran a simple test
job, two IEFBR14 steps, the first allocating a dataset and second
deleting the dataset, thus doing absolutely now I/O to the dataset. In
http://wotho.ethz.ch/RAKF_Problem.zip
<http://wotho.ethz.ch/RAKF_Problem.zip>  you'll find the following
files:

    * juergent.jcl: JCL of the test job
    * juergent_normal.txt: Joblog of a successful run of that job
    * juergent_failure.txt: Joblog of a failed run of that job
    * hardcopy.log: MVS log of the following sequence:

    * IPL, without starting RAKF
    * Logon TSO user juergen and submitting the test job 4 times. All
runs are successfull.
    * Logoff TSO user juergen.
    * Start RAKF.
    * Logon TSO User juergen.
    * Submit test job 3 times: First time the job succeeds, next two
times the job fails.
    * $PI1, $SI1 to restart the initiator.
    * Submit test job 2 times: First time it succeeds, second time it
fails. These two runs are the above mentioned juergent_normal.txt and
juergent_failure.txt
    * Shutdown

I think, this answers the questions regarding the JCL and the properties
of the dataset. The problem always occurs with this sequence, regardless
whether the dataset is written to and what DCB is used, or if it gets
just allocated with no defined DCB.

Regarding your parmlib question: I only modified LNKLST and IEAAPF to
include additional libraries. I did install the following components on
the original turnkey system:

    * Usermods ZP60002A, ZP60005A, ZP60007, ZP60008, ZP60009, ZP60011,
ZP60012, ZP60013, ZP60014, ZP60015, ZP60016, ZP60017, ZP60018, ZP60019,
ZP60020, ZP60021, ZP60022, ZP60023, ZP60024, ZP60025
    * XMIT370, RECV370, IND$FILE
    * latest versions of IMON370, QUEUE, SPY, RPF
    * RAKF

I hope this gives some input to sort out the problem and would
appreciate any idea, what to do to eliminate the problem.

Cheers, Juergen

[halfmeg]

> Juergen wrote:

><snip>

> thanks for your reply and sorry for my late follow up... I was
> three weeks in holidays (Yosemite proved to be more fascinating
> than MVS  [;)

> I'm still convinced, that it's a RAKF problem: I now ran a simple
> test job, two IEFBR14 steps, the first allocating a dataset and
> second deleting the dataset, thus doing absolutely now I/O to the
> dataset.

><snip>

> I hope this gives some input to sort out the problem and would
> appreciate any idea, what to do to eliminate the problem.

I have downloaded your test and will try to reproduce the problem.

Phil

[juergen.winkelmann]

--- In H390-MVS@..., "halfmeg" <opplr@...> wrote:
>
> I have downloaded your test and will try to reproduce the problem.
>
> Phil
>
Just another observation in this context:

I wanted to take a look at the dump thrown by the initiator abend. Thus
I initialized the IPCS problem directory, dataset directory and local
dump directory VSAM clusters according to the sample job mentioned in
http://tech.groups.yahoo.com/group/H390-MVS/message/278
<http://tech.groups.yahoo.com/group/H390-MVS/message/278> . Here is the
IDCAMS output from this job (shortened):

   DEFINE CLUSTER( NAME(JUERGEN.DUMPDIR) UNIQUE +
....
  IDC0508I DATA ALLOCATION STATUS FOR VOLUME PUB002 IS 0
  IDC0509I INDEX ALLOCATION STATUS FOR VOLUME PUB002 IS 0
  IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0

   DEFINE CLUSTER( NAME(IPCS.PROBLEM.DIRECTRY) UNIQUE +
....
  IDC0508I DATA ALLOCATION STATUS FOR VOLUME PUB002 IS 0
  IDC0509I INDEX ALLOCATION STATUS FOR VOLUME PUB002 IS 0
  IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0


   DEFINE CLUSTER( NAME(IPCS.DATA.SET.DIRECTRY) UNIQUE +
....
  IDC3034I COMMAND FAILED DUE TO DUPLICATE RACF PROFILE
  IDC3009I ** VSAM CATALOG RETURN CODE IS 114 - REASON CODE IS IGG0CLDD-0
  IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12

So, again an allocation problem in RAKF context. And of course the same
behaviour as with the initiator problem: Running the job without RAKF
being active, all three clusters get allocated without any problem...

Finally, what's in the dump:

IKJ56650I TIME-03:44:02 PM. CPU-00:00:00 SERVICE-464 SESSION-04:18:32
SEPTEMBER 27,2009
BLS18122I INITIALIZATION IN PROGRESS
BLS18124I TITLE=IEC999I IFG0RR0A,........,INIT    ,        ,WORKAREA =
9A2850
BLS18123I 262 RECORDS, 1,075,248 BYTES, IN THE DUMP

ST

SYSTEM STATUS:
NUCLEUS MEMBER NAME: IEANUC01
TIME OF DAY CLOCK: C4D7D7FB 79F74000  09/26/09 20:58:49 LOCAL
TIME OF DAY CLOCK: C4D7BD29 79F74000  09/26/09 18:58:49 GMT

CPU STATUS:
PSW=070C1000 00E325C6
ASCB009 AT FEC4B8., JOB(JUERGENT), IS CURRENT
ASXB009 AT 9CF300. AND TCB009AD AT 9CC818. ARE CURRENT

SCAN

BLS18058I ERRORS DETECTED IN STRUCTURE(ASCB) AT ASID(1) FECBA0.:
BLS22020I   ASCBASCB NOT EQUAL C'ASCB'
BLS18058I ERRORS DETECTED IN STRUCTURE(ASCB) AT ASID(1) FECE28.:
BLS22026I   LOCATOR ASCBFWDP=00FECBA0. IT MAY BE DAMAGED
BLS18076I 10 PROBES WERE EFFECTIVE

SUMM JOB(JUERGENT)

BLS18058I ERRORS DETECTED IN STRUCTURE(ASCB) AT ASID(9) FECE28.:
BLS22026I   LOCATOR ASCBFWDP=00FECBA0. IT MAY BE DAMAGED
BLS18017I JOB JUERGENT, ASCB009 AT 00FEC4B8
....

I didn't dig deeper, but on first sight we have here some sort of
overwritten memory...

Cheers, Juergen

[halfmeg]

> Juergen wrote:

> > halfmeg wrote:

> > I have downloaded your test and will try to reproduce the problem.

> Just another observation in this context:
 
> I wanted to take a look at the dump thrown by the initiator abend.

><snip>

The 1st difference in results is, I don't get an abended initiator.  I do get a JCL error on the 2nd run however.  Am looking into LIBs loaded on my system vs the RAKF.aws file.  Work to get RAKF for 3.8j was done by Phil Dickinson working with ESG Security System, file CBT165 from CBTTAPE.

Phil

[halfmeg]

> halfmeg wrote:

><snip>
 
> The 1st difference in results is, I don't get an abended initiator.

><snip>

I take that back, the initiator does send a dump to the dump data set.

Looking at your hardcopy log, I see where a stop, then start of the initator permits another good run of the job.

I took the delete step off, ran the IEFBR14 to create the file, deleted it via TSO session, ran the IEFBR14 again and Initator dumped.

I stopped/started the initator, ran the IEFBR14 to create the file, stopped/started the initator again, deleted the file via TSO session, ran the IEFBR14 to create the file and all was fine.

Your comment about overwritten memory sounds valid, however I don't think the source for RAKF was ever uploaded, so unless Phil D. steps in to assist or provide the source he worked with, the base source reverts back to CBT165.

Phil - this isn't the only add-on for which source isn't around

[juergen.winkelmann]

--- In H390-MVS@... <mailto:H390-MVS@...> ,
"halfmeg" <opplr@...> wrote:
>
> Your comment about overwritten memory sounds valid, however I don't
think the source for RAKF was ever uploaded, so unless Phil D. steps in
to assist or provide the source he worked with, the base source reverts
back to CBT165.
>
> Phil - this isn't the only add-on for which source isn't around
>
Hi Phil,

thanks for your efforts... so at least it's not a problem specific to my
system ;-) I absolutely agree with you, that we now probably will not be
able to get further without the help of Phil D. or at least his source.

On the other hand it would be interresting to know, if the problem
already existed when using RAKF on the unmodified Turnkey 3 system,
which was the one used or at least recommended by Phil D., if I
understand the history correctly. If it didn't exist on TK3, then one of
the applied modifications brought it in. In the case of my system these
are basically the ZP6xxxxx Usermods from Greg Price (which of course
wouldn't mean, that it didn't exist before applying the usermods, it may
then simply have overwritten less important memory ;-). Did you test on
the original TK3  or on your TK3 Update system?

Cheers, Juergen

[halfmeg]

> Juergen wrote:

> Hi Phil,
 
> thanks for your efforts... so at least it's not a problem specific
> to my system ;-) I absolutely agree with you, that we now probably
> will not be able to get further without the help of Phil D. or at
> least his source.

There is a possibility the problem isn't in RAKF, but in the SAF additions to TK3 which were added by PTFs.  Perhaps the one Phil D. mentions as a requirement.

> On the other hand it would be interresting to know, if the problem
> already existed when using RAKF on the unmodified Turnkey 3 system,
> which was the one used or at least recommended by Phil D., if I
> understand the history correctly.  If it didn't exist on TK3, then
> one of the applied modifications brought it in. In the case of my
> system these are basically the ZP6xxxxx Usermods from Greg Price
> (which of course wouldn't mean, that it didn't exist before applying
> the usermods, it may then simply have overwritten less important
> memory ;-). Did you test on
> the original TK3  or on your TK3 Update system?

I tested on both, after failure on the Updated environment, I installed original TK3, installed RAKF, tested jobs via card reader.  1st job successful, 2nd job causes init to dump to dump dataset and JCLs.

Greg's Usermods shouldn't have any effect on this to my knowledge.

Phil - busy with real life - http://www.softlib.org/tree.jpg

[Phil Dickinson]

This very much sounds like my problem.
I am particularly intrigued by the VSAM error..... I never managed to get
anywhere with VSAM, I'd even assumed the correct hooks weren't in 3.8j for
VSAM, so this is actually a good thing...LOL

I haven't had a working hercules system for quite some time, years possibly.
 My old system, including RAKF and everything i'll need to trace the problem
is in a packing crate (I've moved).

It was always my intention to clean up the source and return it to the
public domain.... now is the time to do that me thinks, but it will take a
little time, I have a few priority tasks on the list ahead of it.

If you can't wait, the original system is still on CBT.... it wasn't all
that difficult to modify it to work with 3.8j..... I mean if I did it, it
couldn't have been difficult.

Phil - The other phil



>