Embedded gurus,
Can some one advice me how to create relocatable application in C?We
are currently working in an application,where we are supposed to test
the entire RAM area for physical errors in the chip.We have created
our application for this.The problem we are facing is this:

Our application sits in a particular location in RAM.The requirement
states we have to test entire area of RAM including the location where
the code is sitting.(Code in this sense,entire binary image which
includes code,data,stack,bss).So we came up with the idea of testing
the areas other then where our image sits and then move our image to
different location with in RAM which is already tested and then come
back to original location.

But the problem with this solution is ,we are not aware how this
relocation can be done?Does this relocation require compiler
support?In the linker options used in our linker,we have options to
create relocatable modules.But my colleague says it relocates only the
code not the data and section areas.When I mean "relocatable module"
what does it generally mean?

Now whats the way to do this relocation incase my compiler does not
provide support for creating relocatable image?

Looking farward for all your replys and advanced thanks for the same,
Regards,
s.subbarayan.

From:	Stephen Pelc
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 10:09:45 GMT

On 18 Jan 2005 20:48:49 -0800, s_subbarayan@rediffmail.com
(s.subbarayan) wrote:

>Our application sits in a particular location in RAM.The requirement
>states we have to test entire area of RAM including the location where
>the code is sitting.(Code in this sense,entire binary image which
>includes code,data,stack,bss).So we came up with the idea of testing
>the areas other then where our image sits and then move our image to
>different location with in RAM which is already tested and then come
>back to original location.

We've had to write relocatable binaries on systems that do not
well support position independent code. Our solution is most
easily implemented using a few lines of assembler for some code
that calculates the base address at which the code is actually
running.

We compile/link for two different addresses, and produce a data
table that holds the addresses of the differences. This table is added
to the binary. A small piece of start up code applies fixes.
We usually write this start up code in assembler, and it will
require the link address.

From our point of view, this technique is particularly attractive
because it insulates us from many of the details of the final
executable format if it is not a simple memory image.

Stephen

--
Stephen Pelc, stephenXXX@INVALID.mpeltd.demon.co.uk
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeltd.demon.co.uk - free VFX Forth downloads

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 12:50:04 -0700

CBFalconer writes:
> I thought the 360 architecture handled relocation very simply from
> the beginning, with base registers.

standard instructions used register address plus 12bit
displacement. however os/360 compilers and assemblers generated
intermediate executables that had instructions and data intermixed
.... including address constants that conventional programs would load
into registers for use in "relocation".

there was administrative bookkeeping kept at the end of each such file
that listed the location (within the program) of each such address
constant and some (possibly symbolic) information on how to initialize
the address constant when the program was loaded into memory. This
administrative information was commonly referred to as relocatable
adcons (ADdress Constants) .... but they actually became fixed address
constants at the time the program was loaded into memory for actual
execution (aka the executable image out on disk and the executable
image resident in memory tended to defer at least by the swizzling
that went on for such address constants).

so 360 hardware architecture went to all the trouble of making
instruction execution relocatable ... and then, effectively, the
os/360 software guys took some short cuts and established conventions
making the actually executable images in memory bound to fixed address
location.

tss/360 was a parallel operating system effort that was targeted at
time-sharing (aka TSS .. Time Sharing System) designed for 360/67
.... which was the only 360 with virtual memory capability. tss/360
operating system conventions went a lot further towards trying to make
sure that the executable image on disk was exactly the same executable
image running in memory ... and preserving the relocatable 360
hardware instruction paradigm.

In os/360 operating system convention the "relocatble address
constants' that were sprinkled thru-out the binary image on disk had
to be swizzled as part of brining the image into memory for execution
(and at the same time bound the in-memory image to a specific
address).

TSS/360 had a one-level store, page-mapped orientation ... much more
orientated towards what was on disk and what was in memory would be
identical.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	Nick Maclaren
Subject:	Re: Relocating application architecture and compiler support
Date:	20 Jan 2005 20:51:05 GMT

In article ,
Anne & Lynn Wheeler wrote:
>CBFalconer writes:
>> I thought the 360 architecture handled relocation very simply from
>> the beginning, with base registers.
>
>standard instructions used register address plus 12bit
>displacement. however os/360 compilers and assemblers generated
>intermediate executables that had instructions and data intermixed
>... including address constants that conventional programs would load
>into registers for use in "relocation".

Plus others that did other things, including creating position
independent code - I can't remember if any IBM compilers did that,
but it wouldn't surprise me.

I was only half joking when I said that the answer to "Did OS/360
use technique XXX?" is "yes" :-)

Regards,
Nick Maclaren.

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 13:36:52 -0800

Nick Maclaren wrote:

(snip)

> Plus others that did other things, including creating position
> independent code - I can't remember if any IBM compilers did that,
> but it wouldn't surprise me.

None of the OS/360 compilers that I know about.

> I was only half joking when I said that the answer to "Did OS/360
> use technique XXX?" is "yes" :-)

With the number of different people working on it, and with the
variety of design goals, I agree.

-- glen

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 16:29:42 -0700

glen herrmannsfeldt writes:
> But there is one thing that OS/360 does easily that is hard in other
> systems, which is what the LINK macro does. To take another
> program, load it into the same address space, execute it (with the
> original still addressable) and then return.
>
> I remember TOPS-10, where each program was loaded with virtual
> origin of 0. There was a system for sending files to the print
> queue which involved writing the entire address space to disk,
> running the QUEUE program with an option to execute a program when
> it was done, which would then reload and continue on.
>
> Any system with a fixed load point can't have two programs in the
> same address space at the same time.
>
> In the OS/360 case, besides doing LINK, many routines, especially
> access methods, were directly addressable somewhere in system space.
> Now, there are systems which reserve parts of the address space for
> system and parts for user, but that doesn't help LINK.

basically you could do link-like function in both os/360 as well as
tss/360. the primary difference was that os/360 had to read the
executable file image into memory and then run the address constant
swizzle against that were randomly sprinkled thruout the execution
image.

in the tss/360 scenario ... you just had to memory map some portion of
the virtual address space to the executable file image on disk (and
let the paging operation fetch the pages as needed) ... and all the
address constants needing swizzling were kept in a different
structure.

os/360 had a single real address space orientation so that it was
possible for multiple different processes to share the same executable
image because they all shared the same (single, real) address space.

tss/360 had multiple virtual address spaces .... and for multiple
different processes to share the same exact executable copy ... it
relied on the shared segment infrastructure.

in the os/360 scenario, all program loading was to a unique real
address ... since all processes shared the same, single real address
space (there was no chance for program address conflict ... since each
program could be assigned a unique real address as it was read into
memory).

in the tss/360 scenario, different processes might load & populate
their own virtual address space in difference sequences
.... potentially creating address assignment conflict if there was a
requirement that each application have identically assigned virtual
address across all virtual address spaces.

in the os/360 scenario ... if you had two different processes,
the first LINKed app1, then app2, then app3, and finally app4 and
the second LINKed app2, then app4, then app3, and finally app1 ...
it all fell out in the wash since there was a single global
real address space.

a difference between the os/360 and tss/360 operation, is that os/360
allowed all the address constants (needing position location
swizzling) to be randomly sprinkled thruout the executable
image. tss/360 collected address constants (needing swizzling) into a
different structure.

both could do dynamic process location specific binding at program
load time. however, in the tss/360 scenario ... different processes
could share the same exact executable image at different address
locations ... because the executable image was a separate structure
from the structure of address constants (that required location
specific swizzling).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	Peter Flass
Subject:	Re: Relocating application architecture and compiler support
Date:	Sat, 22 Jan 2005 01:39:56 GMT

Anne & Lynn Wheeler wrote:
> basically you could do link-like function in both os/360 as well as
> tss/360. the primary difference was that os/360 had to read the
> executable file image into memory and then run the address constant
> swizzle against that were randomly sprinkled thruout the execution
> image.
>
So why was TSS considered so slow? (not that OS/360 was any ball of speed).

From:	Nick Maclaren
Subject:	Re: Relocating application architecture and compiler support
Date:	21 Jan 2005 11:27:01 GMT

In article ,
glen herrmannsfeldt wrote:
>Nick Maclaren wrote:
>
>> Plus others that did other things, including creating position
>> independent code - I can't remember if any IBM compilers did that,
>> but it wouldn't surprise me.
>
>None of the OS/360 compilers that I know about.

Thanks. It would almost certainly have been an oddball one if so.
We used a large range of compilers, several of which had been developed
at Cambridge, but including other non-IBM ones. There were also a
large number of student compiler projects that got to the point of
usability.

Writing position independent code was pretty easy, but you had to
separate out the system interfaces and/or write your own macros. IBM's
standard ones often generated ACONs and VCONs.

Regards,
Nick Maclaren.

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 10:25:02 -0800

Nick Maclaren wrote:

(snip)

>>>Plus others that did other things, including creating position
>>>independent code - I can't remember if any IBM compilers did that,
>>>but it wouldn't surprise me.

(I wrote)

>>None of the OS/360 compilers that I know about.

> Thanks. It would almost certainly have been an oddball one if so.
> We used a large range of compilers, several of which had been developed
> at Cambridge, but including other non-IBM ones. There were also a
> large number of student compiler projects that got to the point of
> usability.

It might have been possible in a closed system, where users
couldn't write any assembly code. I do remember CALL/OS, a
timesharing system with BASIC, FORTRAN, and PL/I, but I don't
know at all what it was like internally.

With a convention where some registers only held addresses, and
the rest never did you would know which registers to change on
relocation.

I suppose there should be two kinds of position independent
code. One kind could be executed at any address, even more than
one (virtual) address at the same time. The second could
actually be moved to a different address while executing. It is
the second kind that is more complicated because registers could
hold addresses.

-- glen

From:	Nick Maclaren
Subject:	Re: Relocating application architecture and compiler support
Date:	21 Jan 2005 20:44:29 GMT

In article ,
glen herrmannsfeldt wrote:
>
>>>>Plus others that did other things, including creating position
>>>>independent code - I can't remember if any IBM compilers did that,
>>>>but it wouldn't surprise me.
>
>(I wrote)
>
>>>None of the OS/360 compilers that I know about.
>
>> Thanks. It would almost certainly have been an oddball one if so.
>> We used a large range of compilers, several of which had been developed
>> at Cambridge, but including other non-IBM ones. There were also a
>> large number of student compiler projects that got to the point of
>> usability.
>
>It might have been possible in a closed system, where users
>couldn't write any assembly code. I do remember CALL/OS, a
>timesharing system with BASIC, FORTRAN, and PL/I, but I don't
>know at all what it was like internally.

Eh? Not at all. Every compiler that had an assembler interface
also had a set of documented conventions, which people writing
assembler had to follow. Not a problem.

>With a convention where some registers only held addresses, and
>the rest never did you would know which registers to change on
>relocation.

We aren't talking about moving a task once it had been started,
which I agree is and was very hard, but about position independent
code. I.e. code that could be read into memory, anywhere, without
relocation, and executed. It wasn't hard - just a bit tedious.

I can still describe how it was done, if you are interested.

Regards,
Nick Maclaren.

From:	Stephen Fuld
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 21:19:23 GMT

"Nick Maclaren" wrote in message
news:csrpjd$f3i$1@gemini.csx.cam.ac.uk...

snip

> We aren't talking about moving a task once it had been started,
> which I agree is and was very hard,

Certainly on S/360, but not on other then contemporary architectures. All
that is needed is a protected "system base regiser" which is part of the
program's context, to which the hardware adds all program addresses. Then
all programs can start at zero and go to whatever, and all that is needed if
the OS needs to move a program in mid execution is to change the base
register.

BTDT

--
- Stephen Fuld
e-mail address disguised to prevent spam

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 14:58:59 -0800

Stephen Fuld wrote:
(snip)

> Certainly on S/360, but not on other then contemporary architectures. All
> that is needed is a protected "system base regiser" which is part of the
> program's context, to which the hardware adds all program addresses. Then
> all programs can start at zero and go to whatever, and all that is needed if
> the OS needs to move a program in mid execution is to change the base
> register.

But there is one thing that OS/360 does easily that is hard in
other systems, which is what the LINK macro does. To take
another program, load it into the same address space, execute it
(with the original still addressable) and then return.

I remember TOPS-10, where each program was loaded with virtual
origin of 0. There was a system for sending files to the print
queue which involved writing the entire address space to disk,
running the QUEUE program with an option to execute a program
when it was done, which would then reload and continue on.

Any system with a fixed load point can't have two programs in
the same address space at the same time.

In the OS/360 case, besides doing LINK, many routines,
especially access methods, were directly addressable somewhere
in system space. Now, there are systems which reserve parts of
the address space for system and parts for user, but that
doesn't help LINK.

-- glen

From:	Stephen Fuld
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 23:52:02 GMT

"glen herrmannsfeldt" wrote in message
news:css1bm$s75$1@gnus01.u.washington.edu...
> Stephen Fuld wrote:
> (snip)
>
>> Certainly on S/360, but not on other then contemporary architectures.
>> All that is needed is a protected "system base regiser" which is part of
>> the program's context, to which the hardware adds all program addresses.
>> Then all programs can start at zero and go to whatever, and all that is
>> needed if the OS needs to move a program in mid execution is to change
>> the base register.
>
> But there is one thing that OS/360 does easily that is hard in
> other systems, which is what the LINK macro does. To take another
> program, load it into the same address space, execute it (with the
> original still addressable) and then return.

In the interests of simplicity, I left out a few things. In the first
implementation that I know about, there were two "base registers", typicaly
one for instructions and one for data. There was an instruction to "link"
to another chunk of code. The typical mechanism for things like access
methods, was to keep the same data area and link to the code, replacing the
user program code temporarily. Thus the access method code could access the
user program data just fine. When it was done with whatever it had to do,
ir executed the instruction to return to the user code and replace the base
address with the original one. Later models added a second set of two base
registers that allowed further flexibility for more complex schemes. Even
later ones had more features that moved the architecture more toward a
segment scheme (with up to 16 active segments out of something like 32K
total available) that is totally independent of the (later introduced)
paging mechansim.

> I remember TOPS-10, where each program was loaded with virtual origin of
> 0. There was a system for sending files to the print queue which involved
> writing the entire address space to disk, running the QUEUE program with
> an option to execute a program when it was done, which would then reload
> and continue on.

Yuck! That sounds awfull. It seems far simpler for the program to write
the file to disk and simply pass a file name to the printer writer. But I
am sure there were other constraints that I just don't know about.

> Any system with a fixed load point can't have two programs in the same
> address space at the same time.

Actually it can, but it takes some doing, like the hardware I talked about
above, or a convention about using different parts of the address space.
With the extra hardware, it isn't much of a problem.

One extra advantage of the scheme I an taliking abouit is that there is no
need for the "address swizzling stuff" on load that was required by OS/360.
That speeded up program loads.

> In the OS/360 case, besides doing LINK, many routines, especially access
> methods, were directly addressable somewhere in system space. Now, there
> are systems which reserve parts of the address space for system and parts
> for user, but that doesn't help LINK.

Right. See above.

--
- Stephen Fuld
e-mail address disguised to prevent spam

From:	jmfbahciv at aol.com
Subject:	Re: Relocating application architecture and compiler support
Date:	Sat, 22 Jan 05 13:23:08 GMT

In article ,
"Stephen Fuld" wrote:
>
>"glen herrmannsfeldt" wrote in message
>news:css1bm$s75$1@gnus01.u.washington.edu...

>> I remember TOPS-10, where each program was loaded with virtual origin of
>> 0. There was a system for sending files to the print queue which
involved
>> writing the entire address space to disk, running the QUEUE program with
>> an option to execute a program when it was done, which would then reload
>> and continue on.
>
>Yuck! That sounds awfull.

Nope. Glen is misremembering. There was a way to "dump" your
core to disk and then our spooling feature would send it to
the print queue. This often happened, under our batch system.
But I don't know a single person who ever did this command on
purpose :-).

> .. It seems far simpler for the program to write
>the file to disk and simply pass a file name to the printer writer.

Glen was talking about a feature of the -10 in that all
files with a LPT extension, as in FOO.LPT, would automatically
be queued when the job logged out.

/BAH

Subtract a hundred and four for e-mail.

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Sun, 23 Jan 2005 14:08:56 -0800

jmfbahciv@aol.com wrote:

(snip)

> Glen was talking about a feature of the -10 in that all
> files with a LPT extension, as in FOO.LPT, would automatically
> be queued when the job logged out.

I thought there was a way for Fortran programs to send
jobs to the printer while they were still running, and that
the program would have to run QUEUE to do it.

A story I do remember is that QUEUE had an option to run a
desired program when it was finished, I thought to implement
the system previously described. QUEUE also had the ability
to run while a user was not logged in so one could check queue
status without logging in. Someone discovered that the
"run after" option still worked even when a user wasn't logged
in, and that it was run under a privileged account. Submitting
the bug report, there is a box for publish/don't publish and
the person, not expecting it to happen, checked the publish box.

After DEC did publish the bug report they then had to very
quickly fix the bug that allowed people to run privileged programs
without needing to log in. Maybe that is also an urban
legend, but the source seemed pretty reliable.

-- glen

From:	Nick Maclaren
Subject:	Re: Relocating application architecture and compiler support
Date:	22 Jan 2005 13:45:27 GMT

In article ,
Stephen Fuld wrote:
>"glen herrmannsfeldt" wrote in message
>news:css1bm$s75$1@gnus01.u.washington.edu...
>> Stephen Fuld wrote:
>>
>>> Certainly on S/360, but not on other then contemporary architectures.
>>> All that is needed is a protected "system base regiser" which is part of
>>> the program's context, to which the hardware adds all program addresses.
>>> Then all programs can start at zero and go to whatever, and all that is
>>> needed if the OS needs to move a program in mid execution is to change
>>> the base register.

Eh? You are confusing two things. If that refers to physical memory,
then that is just virtual memory support. Not a problem.

If it refers to virtual memory, it won't work for any program that
takes the address of code and needs to compare it with others. In C,
for example, you can take the address of functions and manipulate them
ad lib. You can't relocate them within virtual memory thereafter.

>> But there is one thing that OS/360 does easily that is hard in
>> other systems, which is what the LINK macro does. To take another
>> program, load it into the same address space, execute it (with the
>> original still addressable) and then return.

Yes. It is useful and can be (and has been) implemented even in
variants of Unix. It needs no extra hardware support.

>In the interests of simplicity, I left out a few things. In the first
>implementation that I know about, there were two "base registers", typicaly
>one for instructions and one for data. ...

All numbers of base registers from 1 to 3 were common, and more were
not unknown. If I recall, I once implemented a language system with
over 3, but can't now remember exactly why. Something like the data
for the program and run-time system being separate.

Regards,
Nick Maclaren.

From:	Stephen Fuld
Subject:	Re: Relocating application architecture and compiler support
Date:	Sun, 23 Jan 2005 07:37:55 GMT

"Nick Maclaren" wrote in message
news:cstldn$bug$1@gemini.csx.cam.ac.uk...
> In article ,
> Stephen Fuld wrote:
>>"glen herrmannsfeldt" wrote in message
>>news:css1bm$s75$1@gnus01.u.washington.edu...
>>> Stephen Fuld wrote:
>>>
>>>> Certainly on S/360, but not on other then contemporary architectures.
>>>> All that is needed is a protected "system base regiser" which is part
>>>> of
>>>> the program's context, to which the hardware adds all program
>>>> addresses.
>>>> Then all programs can start at zero and go to whatever, and all that is
>>>> needed if the OS needs to move a program in mid execution is to change
>>>> the base register.
>
> Eh? You are confusing two things. If that refers to physical memory,
> then that is just virtual memory support. Not a problem.

Well, it wasn't virtual memory in the conventional sense. There was no
paging, and you couldn't define a single program larger than the physical
memory (without doing the link type stuff). In a sense, it was "inverted"
virtual memory, or perhaps "virtual addresses" in that wheras in virtual
memory you could have addresses larger than physical memory that were mapped
to the same physical address at different times, in this scheme, you had
multiple pieces of code with the same "virtual" address mapped to the same
physical address at different times.

> If it refers to virtual memory, it won't work for any program that
> takes the address of code and needs to compare it with others. In C,
> for example, you can take the address of functions and manipulate them
> ad lib. You can't relocate them within virtual memory thereafter.

Remember, the base register was invisible to the program. If a program
loaded an address into a register, or passed it as an argument, it was
always a program relative address. If the address was used to address
physical memory (either via a load/store or a jump/call) the hardware added
the base register contents to it at that time.

--
- Stephen Fuld
e-mail address disguised to prevent spam

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Sun, 23 Jan 2005 12:28:46 -0800

Stephen Fuld wrote:

(snip of S/360 and base displacement addressing)

> Remember, the base register was invisible to the program. If a program
> loaded an address into a register, or passed it as an argument, it was
> always a program relative address. If the address was used to address
> physical memory (either via a load/store or a jump/call) the hardware added
> the base register contents to it at that time.

Did you ever write C programs for the original Macintosh?
(I didn't, either, but I read some of the manuals.)

Many routines would, instead of pointers return pointers to pointers
such that the OS could move things around when necessary.
(The 68000 didn't have an MMU.) There were complicated rules
on the use of pointers because you had to know when the OS could
move things and make sure you didn't keep a pointer to something
that could be moved.

With OS/360 once you load an address into a register that
object can't move because the system doesn't know to change
the register.

-- glen

From:	Stephen Fuld
Subject:	Re: Relocating application architecture and compiler support
Date:	Mon, 24 Jan 2005 02:14:01 GMT

"glen herrmannsfeldt" wrote in message
news:6PydnVOlwMlnlWncRVn-tw@comcast.com...
> Stephen Fuld wrote:
>
> (snip of S/360 and base displacement addressing)
>
>> Remember, the base register was invisible to the program. If a program
>> loaded an address into a register, or passed it as an argument, it was
>> always a program relative address. If the address was used to address
>> physical memory (either via a load/store or a jump/call) the hardware
>> added the base register contents to it at that time.
>
> Did you ever write C programs for the original Macintosh?
> (I didn't, either, but I read some of the manuals.)

No, I never did.

> Many routines would, instead of pointers return pointers to pointers
> such that the OS could move things around when necessary.
> (The 68000 didn't have an MMU.) There were complicated rules
> on the use of pointers because you had to know when the OS could
> move things and make sure you didn't keep a pointer to something
> that could be moved.

That sounds messy. What I was talking about required none of that.
Everything worked naturally, with no special needs to know when the OS could
move things. In fact, the OS could move things pretty much any time that
you could have an interrupt. All pointers were "program relative" within
your program and only got converted to physical memory addresses when used
by the hardware.

> With OS/360 once you load an address into a register that
> object can't move because the system doesn't know to change
> the register.

Right. And that was the cause of much grief among OS/360 users since that
meant no swapping out capability to compact memory or to suspend a program
(unless it got loaded back to the same physical memory location). This
caused reduced throughput as you then had things like fixed memory
partitions (MFT) and even the variable ones (MVT) had limitations. It also
made things like interactive programming (the original TSO) a real dog
(which we have discussed here before).

--
- Stephen Fuld
e-mail address disguised to prevent spam

From:	Peter Flass
Subject:	Re: Relocating application architecture and compiler support
Date:	Sat, 22 Jan 2005 01:38:36 GMT

glen herrmannsfeldt wrote:

> Stephen Fuld wrote:
> (snip)
>
>> Certainly on S/360, but not on other then contemporary architectures.
>> All that is needed is a protected "system base regiser" which is part
>> of the program's context, to which the hardware adds all program
>> addresses. Then all programs can start at zero and go to whatever,
>> and all that is needed if the OS needs to move a program in mid
>> execution is to change the base register.
>
>
> But there is one thing that OS/360 does easily that is hard in
> other systems, which is what the LINK macro does. To take another
> program, load it into the same address space, execute it (with the
> original still addressable) and then return.
>
> I remember TOPS-10, where each program was loaded with virtual origin of
> 0. There was a system for sending files to the print queue which
> involved writing the entire address space to disk, running the QUEUE
> program with an option to execute a program when it was done, which
> would then reload and continue on.
>
> Any system with a fixed load point can't have two programs in the same
> address space at the same time.
>
> In the OS/360 case, besides doing LINK, many routines, especially access
> methods, were directly addressable somewhere in system space. Now,
> there are systems which reserve parts of the address space for system
> and parts for user, but that doesn't help LINK.
>

I was going to say something like this, but then I reread Stephen's
comments about the system-base-register being part of the program's
context. Possibly he was thinking of the equivalent of LINK macro
resetting the register for each call/return. Then, of course, it blows
away any access to the other program's data.

From:	pc
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 21:41:26 GMT

Stephen Fuld wrote:

> "Nick Maclaren" wrote in message
> news:csrpjd$f3i$1@gemini.csx.cam.ac.uk...
>
> snip
>
>
>>We aren't talking about moving a task once it had been started,
>>which I agree is and was very hard,
>
>
> Certainly on S/360, but not on other then contemporary architectures. All
> that is needed is a protected "system base regiser" which is part of the
> program's context, to which the hardware adds all program addresses. Then
> all programs can start at zero and go to whatever, and all that is needed if
> the OS needs to move a program in mid execution is to change the base
> register.
>
> BTDT
>

i guess all those 'adcons' and the expectation that appl'n programs must
anticipate interrupts were the real complication. the base-displacement
scheme seemed clever to me, but BAL was my first language so maybe i
just didn't know any better. didn't have much to do with Cobol, but i
remember seeing these giant programs that were written for 360 DOS and
depended on a bit of link-editor support. feature referred to as some
kind of 'overlay'. i vaguely recall that they could thrash away for
hours, doing the equivalent of an OS 'xctl' several times a second.

but i think the original idea must have been far-sighted for its time
and probably under-exploited, just like Gene Amdahl's TRT. had a chance
to meet him when he was in his seventies, still working and answering
the phones when the receptionist wasn't around. passed it up partly
because i thought he deserved more respect than listening to me ask why
cpu's didn't have double-ended stacks. in spite of all the fooferah
that got added by the operating systems, what he and others did still
seems very clear and clean to me. not like all that little-endian Intel
stuff which i gather has complicated everybody's life in unseen ways
just so some 8080 programs could keep working.

a boss once made us read a business article about how complicated jet
engines are when they don't really need to be if we forget about
creature comforts for passengers and municipal bylaws. i guess 360's
were a bit like this - you had to air condition them, but they also had
heaters to warm up the core!

pc

From:	jmfbahciv at aol.com
Subject:	Re: Relocating application architecture and compiler support
Date:	Sat, 22 Jan 05 13:15:32 GMT

In article ,
glen herrmannsfeldt wrote:
>Nick Maclaren wrote:
>
>(snip)
>
>>>>Plus others that did other things, including creating position
>>>>independent code - I can't remember if any IBM compilers did that,
>>>>but it wouldn't surprise me.
>
>(I wrote)
>
>>>None of the OS/360 compilers that I know about.
>
>> Thanks. It would almost certainly have been an oddball one if so.
>> We used a large range of compilers, several of which had been developed
>> at Cambridge, but including other non-IBM ones. There were also a
>> large number of student compiler projects that got to the point of
>> usability.
>
>It might have been possible in a closed system, where users
>couldn't write any assembly code. I do remember CALL/OS, a
>timesharing system with BASIC, FORTRAN, and PL/I, but I don't
>know at all what it was like internally.
>
>With a convention where some registers only held addresses, and
>the rest never did you would know which registers to change on
>relocation.
>
>I suppose there should be two kinds of position independent
>code. One kind could be executed at any address, even more than
>one (virtual) address at the same time. The second could
>actually be moved to a different address while executing. It is
>the second kind that is more complicated because registers could
>hold addresses.

It can't be moved while executing because the monitor is the
one whose code is getting executed to do that shuffling task.

/BAH

Subtract a hundred and four for e-mail.

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Sun, 23 Jan 2005 14:10:34 -0800

jmfbahciv@aol.com wrote:

(snip, I wrote)

>>I suppose there should be two kinds of position independent
>>code. One kind could be executed at any address, even more than
>>one (virtual) address at the same time. The second could
>>actually be moved to a different address while executing. It is
>>the second kind that is more complicated because registers could
>>hold addresses.

> It can't be moved while executing because the monitor is the
> one whose code is getting executed to do that shuffling task.

Well, that was supposed to include during interrupts.

-- glen

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 19:39:56 -0700

Peter Flass writes:
> So why was TSS considered so slow? (not that OS/360 was any ball of
> speed).

lots of reasons ... long path lengths ... over complex algorithms
.... bloated code size.

lets say you had os/360 (or cms) environment that did program load
.... it could queue up a single i/o that read application into memory
in multiple 64k chucks ... all batched together.

tss/360 could have a large compiler laid out as 1mbyte memory mapped
file ... do a memory mapping for 256 4k pages ... and then possibly
individually page fault each 4k page one at a time. To some extent
they got over enamored with concept of one-level store and paging and
lost the fact that the purity of the concept could significantly
increase the latency if all transfers had to be serialized 4k bytes at
a time (as opposed to doing program loading batching in larger
transfer units).

kernel was really large ... tss/360 supposedly was going to have
target of 512k 360/67 ... but quite quickly grew to minimum 768k
storage because of bloated fixed kernel. at one point there was a
statement about a set of benchmarks down on a 1mbyte 360/67
uniprocessor and on a 2mbyte 360/67 two processor machine ... with the
2mbyte/2processor having 3.8 times the thruput of the single processor
benchmark. the official coment was (while the uniprocessor system was
really slow), the two processor benchmark having 3.8 times the thruput
of the single processor benchmark demonstrated how advanced the
tss/360 multiprocessor algorithms were (getting almost four times the
thruput with only twice the hardware). the actual explanation was that
the kernel was so bloated that it could hardly fit in 1mbyte
configuration .... and that in the 2mbyte configuration there was
almost enuf memory (not in use by the kernel) for running
applications.

benchmark at the university on 768k 360/67 running tss/360 .. i
believe prerelease 0.68 with four emulated users doing mix-mode
fortran edit, compile and execute ... had multi-second response for
trivial program edit line input. at the same time, on the same
hardware with cp/67 running same mix-mode fortran edit, compile and
execute had subsecond response for trivial edit line input .... but
running 30 conccurent cms users ... compared to 4 tss users (although
i had already done a lot of cp/67 performance enhancements).

there was folklore that when tss/360 was decommited and the
development group reduced from 1200 people to possibly 20 ... that a
single person now had responsibility for the tss/360 scheduler and a
large number of other modules. supposedly the person discovered that
on a pass thru the kernel ... every involved kernel module was
repeatedly calling the scheduler ... when it turn out that it was only
necessary to call the scheduler once per kernel call (rather than
every module calling the scheduler resulting in multiple scheduler
calls per kernel call). fixing that is claimed to have eliminated a
million(?) instructions per kernel call.

at some point in the tss/370 life ... it had been relatively stable
for a long time ... with a lot work over the years on performance
tweaking ... they claimed that they got the total pathlength to handle
page fault (page fault, page replacement select, schedule page read,
task switch, page read complete, task switch, etc) down to maybe five
times better than MVS ... but still five times longer than my
pathlength for equivalent sequence in vm/370.

it was in this era (late 70s) that they did the unix project for at&t
.... where a kernel semantics interface was built on low-level tss/370
kernel function to support high-level unix environment. i believe this
tss/370 environment for unix was called ssup.

one of the early tss/360 features that i gave them trouble about was
the whole thing about interactive vs-a-vs batch. early 360/67 had 2301
"drum" for limited amount of high-speed paging space and the rest was
2311 (very slow) disks. the scenario went something like ... if some
finished a line in edit and hit return (on 2741) ... tss kernel would
recognize it as interactive and pull the pages that the task had been
using the previous time from 2311 into memory and then write them all
out to 2301 ... once that was done it would start the task ... which
would allow it to page fault the pages back into memory from the 2301.
when the trivial interactive task went into wait (for more terminal
input), the kernel would pull all of the tasks pages off the 2301 back
into memory and move them to 2311. This dragging of pages off 2311 int
memory and out to 2301 and then dragging the pages off 2301 into
memory and out to 2311 would occur regardless of whether there was any
contention for either 2301 space or real memory space. It was a neat,
fancy algorithm and they would do it everytime ... whether it was
necessary or not ... just because it was so neat(?).

.... there is a lot more ... but that should be enuf for the moment.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	Dennis Ritchie
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 06:21:16 -0000

"Anne & Lynn Wheeler" wrote in message news:m3vf9r7udv.fsf@lhwlinux.garlic.com...
> CBFalconer writes:
> > I thought the 360 architecture handled relocation very simply from
> > the beginning, with base registers.
>
> standard instructions used register address plus 12bit
> displacement. however os/360 compilers and assemblers generated
> intermediate executables that had instructions and data intermixed
> ... including address constants that conventional programs would load
> into registers for use in "relocation".
....

followed by an explanation about relocation techniques
for the /360.

Corby wrote a paper
Corbató, F. J., System requirements for multiple-access, time-shared computers,
MAC-TR-3, May 1964.

"It is now clear that it is possible to create a general-purpose time-shared
multiple access system on most contemporary computers.
However, it is equally clear that none of the existent
computers are well designed for multiple access systems...."

essentially explaining why the /360 architecture wouldn't do for
what was becoming the Multics project. There's a downloadable
version referenced in the Multicians' bibliography at
http://www.multicians.org/biblio.html (the paper is image format).

The do-it-yourself relocation scheme caused lots of pain for
more vanilla users too: ultimately the addresses were physically
absolute and the OS doesn't know where the addresses are.
So if the operator wanted to swap something out, it was necessary
to bring it back to the same place.

Base-and-bounds relocation (added as a hack by Boeing, as
Wheeler reported) and of course the 360/67 and later DAT fixed
this.

Dennis

From:	John R. Levine
Subject:	Re: Relocating application architecture and compiler support
Date:	21 Jan 2005 01:58:13 -0500

>followed by an explanation about relocation techniques for the /360.

>The do-it-yourself relocation scheme caused lots of pain for more
>vanilla users too: ultimately the addresses were physically absolute
>and the OS doesn't know where the addresses are. So if the operator
>wanted to swap something out, it was necessary to bring it back to
>the same place.

Yup. IBM published a most interesting set of articles on the S/360 in
the IBM Systems Journal in 1964. An article by Gene Amdahl on
"Processing unit design considerations" says on p 147-148 that
relocation hardware was expensive, so they designed the 360's
instruction set with small address displacements in the instructions
to force all addressing to be relative to base registers so you can
relocate programs by changing the base registers.

http://www.research.ibm.com/journal/sj/032/amdahl.pdf

By the following year they'd figured out that didn't work. In an
article "On dynamic relocation" W. C. McGee, which references
Corbató's report, he covers the various ways to make programs
relocatable and concludes that you need hardware like that in
the just-announced 360/67.

http://www.research.ibm.com/journal/sj/043/ibmsjIVRIIIC.pdf

From:	CBFalconer
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 10:58:59 GMT

"John R. Levine" wrote:
> Dennis Ritchie wrote:
>
>>followed by an explanation about relocation techniques for the /360.
>
>> The do-it-yourself relocation scheme caused lots of pain for more
>> vanilla users too: ultimately the addresses were physically absolute
>> and the OS doesn't know where the addresses are. So if the operator
>> wanted to swap something out, it was necessary to bring it back to
>> the same place.
>
> Yup. IBM published a most interesting set of articles on the S/360
> in the IBM Systems Journal in 1964. An article by Gene Amdahl on
> "Processing unit design considerations" says on p 147-148 that
> relocation hardware was expensive, so they designed the 360's
> instruction set with small address displacements in the instructions
> to force all addressing to be relative to base registers so you can
> relocate programs by changing the base registers.
>
> http://www.research.ibm.com/journal/sj/032/amdahl.pdf
>
> By the following year they'd figured out that didn't work. In an
> article "On dynamic relocation" W. C. McGee, which references
> Corbató's report, he covers the various ways to make programs
> relocatable and concludes that you need hardware like that in
> the just-announced 360/67.
>
> http://www.research.ibm.com/journal/sj/043/ibmsjIVRIIIC.pdf

In the 1974..78 period I developed my own p-code system. It was
designed to run on at least the 8080, and was largely patterned
after the HP3000 architecture. Code was loaded in segments, and
those segments were never altered. Data was stack (and stack
marker) relative. Unfortunately data could not be moved, due to
the lack of hardware, which was a limitation _on the 8080_.

Code segments limited branching to functions, and the code for
branches, conditional or otherwise, was always self-relative.
Function call was done via a table appended to the segment, and
addressed by table index. The table entries could be negative,
when they specified a self-relative point within the segment, or
positive, when they contained two fields in 0..127 (leaving a spare
bit). These described segment number, and an index within that
segments transfer table, and provided for inter-segment calls.
This meant the stack marker on a call had to hold segment and
segment displacement values.

The result was that a segment could be freely moved during any
inter-segment call, and that no more than the single destination
segment needed to be in memory at any one time. A single system
table indexed by segment number sufficed to keep track of
everything. A table required 8 bytes per segment for this, and was
fixed in memory.

The system could execute an upper limit of approximately 3 Mbytes
of code using the swapping algorithms (which were independent of
the organization). For comparison the complete ISO compliant
Pascal compiler occupied about 40 Kbytes. The same compiler,
compiled to machine code for either the HP3000 or 8086 (MsDos),
occupied about 150 Kbytes, which indicates the compactness of the
actual pcode.

I intended the system to be directly portable to many
architectures. When the 8088 effectively took over the PC world
circa 1982 I started to port it, but never finished. The reason
was that there was no need for it in my particular small niche,
which was providing embedded machinery for medical testing, and
revolved around 8080 based hardware I had designed and built. It
all worked so well that I had no excuse for rebuilding it. It had
the unfortunate limitation to 64k of data space, barring which it
would be viable today.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 13:25:05 -0700

glen herrmannsfeldt writes:
> This reminds me of the way Macintosh OS used to work. (Does it
> still do that?) Fortunately I never wrote any low level Mac code,
> but you had to be careful when things could be relocated, and not
> store addresses across system calls that could move things around.
>
> In the code above, you can't move things around between the AR and
> the BALR. The original mac didn't do preemptive multitasking, so
> the user program always knew when things could move and when they
> couldn't.
>
> On the other hand, for a system like 680x0 with separate address and
> data registers, maybe it isn't so hard.

it turns out that this is slightly different operation ... i (re)wrote
the displatcher, scheduler, page replacement, etc for cp/67 ... a lot
of it as undergraduate ... which was shipped in the standard product.
a lot of this was dropped in the morphing of cp/67 into vm/370 ... but
I got to put it (and a lot of other stuff) back in when i did
the resource manager for vm/370
http://www.garlic.com/~lynn/subtopic.html#fairshare
http://www.garlic.com/~lynn/subtopic.html#wsclock

because cp/67 (and vm/370) had virtual memory support ... each user
had their own virtual address space ... where pages could be brought
in and out of real storage at arbitrary real addresses.

the problem i had in the early 70s with location independent code ...
was that i wanted to be able to page-map applications to files on
disks ... and that multiple different virtual address spaces could
share the same exact (r/o) page-mapped information.

the problem with the standard os/360 model was that when executable
files out on disk were mapped into memory (virtual or real), the
program loader had to run thru arbritrary locations in the executable
image ... swizzling arbritrary address constants into different values
(randomly spread thruout the executable image).

as a result, it wasn't just a simple matter of page mapping an
executable image file (out on disk) into a virtual address space
.... there was still all this address constant swizzling that had to be
done before the program could actually start execution. Furthermore,
the default was to touch every possible page that potentially
contained an address constant (that concievably might be used
.... whether it actually was used or not) and do the appropriate
swizzling operation. And even further complicating the process was
that the swizzling operation went on within the specific virtual
address space that had just mapped the executable image.

So the swizzling operation that would go on in each virtual address
space ... pre-touched and changed an arbritrary number of virtual
pages (whether the actually application execution would touch those
specific pages or not) ... as well as making the pages chnaged
.... defeating the ability to have the program image mapped into r/o
shared segments that were possibly common across a large number of
different address spaces. The issue that i was trying to address
wasn't what might be in the registers of each individual process
context in each virtual address space .... it was trying to make the
physical executable storage image of the application r/o shared
concurrently across a large number of different address spaces. Any
modification required on the contents of that executable image,
defeated the ability to have it r/o shared concurrently across a large
number of different address spaces (as well as possibly prefetching
and changing pages that might never actually be used).

So that was the first problem.

A subset of the relocating shared segment implemention (that I had
done in the early 70s) was picked up by the product group and released
under the feature name as DCSS (DisContiguous Shared Segments). I had
done a page mapped file system along with the extended shared segment
support ... so that it was relatively straight-forward to page map
objects in the file system into virtual address spaces. The page
mapped file system wasn't part of the subset of stuff that was picked
up for DCSS. random posts on the page mapped file system work
http://www.garlic.com/~lynn/subtopic.html#mmap

They addressed the address constant swizzling problem by defining
globally unique system addresses for every application that would
reside in predefined virtual memory segments; loading the application
at that predefined virtual memory location and saving the virtual
address space image to a reserved portion of the system-wide paging
system (in part because they had failed to pick up the page mapped
filesystem enhancements).

So DCSS had a new kind of system command that would map a portion
of a virtual address space to a presaved application image (and
specify things like shared segments, etc).

So there were several limitations ... one it required system wide
coordination of the presaved applications as well as system privileges
to setup stuff (i.e. individual department couldn't enable their own
private applications).

A large installation would have more applications defined in this
infrastructure than could fit in a single 16mbyte virtual address
space ... and as a result, there had to be careful management of
applications that were assigned conflicting predefined, preswizzled
virtual addresses. While no single user was likely to try an map all
possible applications into a single address space at the same moment
.... it was possible that a single user might need to map an arbritrary
combination of applications (in total less than 16mbytes), some of
which may have conflicting, pre-assigned and swizzled virtual address
images. As systems got bigger with wider variety of users and
applications, the problem of pre-swizzled virtual application images
with conflicting virtual address locations increased.

So the next solution was to have multiple pre-swizzled application
images defined at multiple different virtual address location. Users
would decide on the combination of page image applications and
libraries that needed to have concurrently loaded ... and try and find
some possible combination of the multiple different images of each
application that could comfortably co-exist in the same address space.

The original implementation that i had done from the ealy 70s ... had
allowed page mapping arbitrary files as virtual address apace images
at arbitrary virtual address locations .... w/o needing to swizzle
address constants embedded in those page mapped images ...
in part because I had both done a lot of work on allowing address
location independent executable images
http://www.garlic.com/~lynn/subtopic.html#adcon
as well as page mapped filesystem enhancements
http://www.garlic.com/~lynn/subtopic.html#mmap

and furthermore that an executable image could occupy a read-only shared
segment ... were the same exact same pages were mapped concurrently
into multiple different virtual address spaces ... at possibly
different virtual addressses.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 18:10:23 -0700

glen herrmannsfeldt writes:
> To do relocation using base registers would require that the OS
> know which registers contained addresses. There is no hardware
> support for that.

however, there was one hardware mechanism of getting a value
of the current instruction into a register for addressability
.... w/o there having been loaded from some sort of address
constant ... standard convention of

BALR R12,0
USING *,R12

assembler convention and could also be used by compilers. Normally,
branch-and-link was used for calls to subroutines with the supplied to
branch address ... and at the same time saving the address of where
the call came from. The above sequence was special case of
branch-and-link, where it didn't actually take any branch but did
establish the current address in a register (w/o requiring any address
constant).

a fairly typical os/360 calling convention is something like:

L R15,=A(subroutine)
BALR R14,R15

or

L R15,=V(subroutine)
BALR R14,R15

where =A(subroutine) and =V(subroutine) become address constants that
are internal to the program. In standard os/360, address constants are
embedded inside the program and administrative detail is added at the
end of the program which the program loader uses to swizzle all
address constants to their absolute value when the program is loaded.

While the standard hardware didn't provide relocation of base
registers .... it did provide the BALR convention of establishing an
address on the fly w/o requiring (fixed) address constant and it
didn't preclude software conventions that avoided the use of fixed
address constants embedded in the executable image of the code.

TSS/360 created a different type of convention where on entry to a
program, certain registers had preloaded values ... one of the
registers contains a pointer to block of storage outside the
executable code image ... where various things like address constants
were located. The other convention was something like this

L R15,=V(subroutine-executablebase)
AR R15,R12
BALR R14,R15

Where R12 contains "executablebase" for situation invoving
displacements larger than 12bits. In os/360 terms, the value
=A(subroutine-executablebase) is an absolute address constant (not a
relocatible address constant requiring swizzle at program load)
..... amd would be the same/constant value, regardless of the address
that the program image was loaded. I used the above convention a lot
in the early 70s for relocable shared segments.

One argument for =V(subroutine) address constant was that there was
effectively "early binding" of the address (performed at program load
time) ... and saved a register (assigning extra register for the
convention) as well as saving an instruction in each calling sequence.

Now there were 360 machines that did have some additional relocation
provided (not the virtual memory kind found in 360/67). System
engineer on boeing account adapted a version of cp/67 (normally using
paged virtual memory) to run on 360/50(?). My understanding is the
360/50 had something that I think was referred to as a DIL instruction
(not part of standard 360 instruction operation) that was used in
conjunction with some emulation packages (7090?) that allowed
specification of base&bound .... basically a base value that the
hardware added to all address calculations and then checked against
the a bound value. This allowed simple contiguous address relocation
(and required swapping of whole address space ... rather than the more
granular paging available on 360/67.

This would be more akin to the current mainframe implementation of
hardware supported virtual machines called LPARS (logical partitions)
where physical memory can be partitioned into logical contiguous
areas. No swapping or paging is supported by the hardware microcode
for LPARs ... just the (relative) static partitioning of available
physical memory. LPARS started out sort of being a significant subset
of the mainframe virtual machine operating system support dropped
into the hardware of the mainframe.

mainframe logical partition getting eal5 certification:
http://www-1.ibm.com/servers/eserver/zseries/security/eal5_ac.html

LPAR support even has extended to non-mainframes:
http://www-1.ibm.com/servers/eserver/iseries/lpar/
http://www-1.ibm.com/servers/eserver/pseries/lpar/
http://www-1.ibm.com/servers/eserver/pseries/lpar/faq_2.html

lots of past posts mentioning LPARs:
http://www.garlic.com/~lynn/98.html#45 Why can't more CPUs virtualize themselves?
http://www.garlic.com/~lynn/98.html#57 Reliability and SMPs
http://www.garlic.com/~lynn/99.html#191 Merced Processor Support at it again
http://www.garlic.com/~lynn/2000b.html#50 VM (not VMS or Virtual Machine, the IBM sort)
http://www.garlic.com/~lynn/2000b.html#51 VM (not VMS or Virtual Machine, the IBM sort)
http://www.garlic.com/~lynn/2000b.html#52 VM (not VMS or Virtual Machine, the IBM sort)
http://www.garlic.com/~lynn/2000b.html#61 VM (not VMS or Virtual Machine, the IBM sort)
http://www.garlic.com/~lynn/2000b.html#62 VM (not VMS or Virtual Machine, the IBM sort)
http://www.garlic.com/~lynn/2000c.html#8 IBM Linux
http://www.garlic.com/~lynn/2000c.html#50 Does the word "mainframe" still have a meaning?
http://www.garlic.com/~lynn/2000c.html#68 Does the word "mainframe" still have a meaning?
http://www.garlic.com/~lynn/2000c.html#76 Is a VAX a mainframe?
http://www.garlic.com/~lynn/2000f.html#78 TSS ancient history, was X86 ultimate CISC? designs)
http://www.garlic.com/~lynn/2000g.html#3 virtualizable 360, was TSS ancient history
http://www.garlic.com/~lynn/2000.html#8 Computer of the century
http://www.garlic.com/~lynn/2000.html#63 Mainframe operating systems
http://www.garlic.com/~lynn/2000.html#86 Ux's good points.
http://www.garlic.com/~lynn/2001b.html#72 Z/90, S/390, 370/ESA (slightly off topic)
http://www.garlic.com/~lynn/2001d.html#67 Pentium 4 Prefetch engine?
http://www.garlic.com/~lynn/2001e.html#5 SIMTICS
http://www.garlic.com/~lynn/2001e.html#61 Estimate JCL overhead
http://www.garlic.com/~lynn/2001f.html#17 Accounting systems ... still in use? (Do we still share?)
http://www.garlic.com/~lynn/2001f.html#23 MERT Operating System & Microkernels
http://www.garlic.com/~lynn/2001h.html#2 Alpha: an invitation to communicate
http://www.garlic.com/~lynn/2001h.html#33 D
http://www.garlic.com/~lynn/2001.html#34 Competitors to SABRE?
http://www.garlic.com/~lynn/2001l.html#24 mainframe question
http://www.garlic.com/~lynn/2001m.html#38 CMS under MVS
http://www.garlic.com/~lynn/2001n.html#26 Open Architectures ?
http://www.garlic.com/~lynn/2001n.html#31 Hercules etc. IBM not just missing a great opportunity...
http://www.garlic.com/~lynn/2001n.html#32 Hercules etc. IBM not just missing a great opportunity...
http://www.garlic.com/~lynn/2002b.html#44 PDP-10 Archive migration plan
http://www.garlic.com/~lynn/2002c.html#53 VAX, M68K complex instructions (was Re: Did Intel Bite Off More Than It Can Chew?)
http://www.garlic.com/~lynn/2002d.html#31 2 questions: diag 68 and calling convention
http://www.garlic.com/~lynn/2002e.html#25 Crazy idea: has it been done?
http://www.garlic.com/~lynn/2002e.html#75 Computers in Science Fiction
http://www.garlic.com/~lynn/2002f.html#6 Blade architectures
http://www.garlic.com/~lynn/2002f.html#57 IBM competes with Sun w/new Chips
http://www.garlic.com/~lynn/2002n.html#6 Tweaking old computers?
http://www.garlic.com/~lynn/2002n.html#27 why does wait state exist?
http://www.garlic.com/~lynn/2002n.html#28 why does wait state exist?
http://www.garlic.com/~lynn/2002o.html#0 Home mainframes
http://www.garlic.com/~lynn/2002o.html#15 Home mainframes
http://www.garlic.com/~lynn/2002o.html#16 Home mainframes
http://www.garlic.com/~lynn/2002o.html#18 Everything you wanted to know about z900 from IBM
http://www.garlic.com/~lynn/2002p.html#4 Running z/VM 4.3 in LPAR & guest v-r or v=f
http://www.garlic.com/~lynn/2002p.html#40 Linux paging
http://www.garlic.com/~lynn/2002p.html#44 Linux paging
http://www.garlic.com/~lynn/2002p.html#54 Newbie: Two quesions about mainframes
http://www.garlic.com/~lynn/2002p.html#55 Running z/VM 4.3 in LPAR & guest v-r or v=f
http://www.garlic.com/~lynn/2002q.html#26 LISTSERV Discussion List For USS Questions?
http://www.garlic.com/~lynn/2003c.html#41 How much overhead is "running another MVS LPAR" ?
http://www.garlic.com/~lynn/2003f.html#56 ECPS:VM DISPx instructions
http://www.garlic.com/~lynn/2003.html#9 Mainframe System Programmer/Administrator market demand?
http://www.garlic.com/~lynn/2003.html#14 vax6k.openecs.org rebirth
http://www.garlic.com/~lynn/2003.html#15 vax6k.openecs.org rebirth
http://www.garlic.com/~lynn/2003.html#56 Wild hardware idea
http://www.garlic.com/~lynn/2003k.html#9 What is timesharing, anyway?
http://www.garlic.com/~lynn/2003l.html#12 Why are there few viruses for UNIX/Linux systems?
http://www.garlic.com/~lynn/2003l.html#41 Secure OS Thoughts
http://www.garlic.com/~lynn/2003m.html#32 SR 15,15 was: IEFBR14 Problems
http://www.garlic.com/~lynn/2003m.html#37 S/360 undocumented instructions?
http://www.garlic.com/~lynn/2003n.html#13 CPUs with microcode ?
http://www.garlic.com/~lynn/2003n.html#29 Architect Mainframe system - books/guidenance
http://www.garlic.com/~lynn/2003o.html#52 Virtual Machine Concept
http://www.garlic.com/~lynn/2004b.html#58 Oldest running code
http://www.garlic.com/~lynn/2004c.html#4 OS Partitioning and security
http://www.garlic.com/~lynn/2004c.html#5 PSW Sampling
http://www.garlic.com/~lynn/2004d.html#6 Memory Affinity
http://www.garlic.com/~lynn/2004e.html#26 The attack of the killer mainframes
http://www.garlic.com/~lynn/2004e.html#28 The attack of the killer mainframes
http://www.garlic.com/~lynn/2004f.html#47 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004g.html#15 Infiniband - practicalities for small clusters
http://www.garlic.com/~lynn/2004k.html#37 Wars against bad things
http://www.garlic.com/~lynn/2004k.html#43 Vintage computers are better than modern crap !
http://www.garlic.com/~lynn/2004m.html#41 EAL5
http://www.garlic.com/~lynn/2004n.html#10 RISCs too close to hardware?
http://www.garlic.com/~lynn/2004o.html#13 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2004o.html#32 What system Release do you use... OS390? z/os? I'm a Vendor S
http://www.garlic.com/~lynn/2004q.html#18 PR/SM Dynamic Time Slice calculation
http://www.garlic.com/~lynn/2004q.html#72 IUCV in VM/CMS

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 10:43:54 -0800

Anne & Lynn Wheeler wrote:

> glen herrmannsfeldt writes:
>
>>To do relocation using base registers would require that the OS
>>know which registers contained addresses. There is no hardware
>>support for that.

> however, there was one hardware mechanism of getting a value
> of the current instruction into a register for addressability
> ... w/o there having been loaded from some sort of address
> constant ... standard convention of

> BALR R12,0
> USING *,R12

But once you do that it can't be relocated without knowing which
registers contain addresses.

(snip)

> TSS/360 created a different type of convention where on entry to a
> program, certain registers had preloaded values ... one of the
> registers contains a pointer to block of storage outside the
> executable code image ... where various things like address constants
> were located. The other convention was something like this

> L R15,=V(subroutine-executablebase)
> AR R15,R12
> BALR R14,R15

> Where R12 contains "executablebase" for situation invoving
> displacements larger than 12bits. In os/360 terms, the value
> =A(subroutine-executablebase) is an absolute address constant (not a
> relocatible address constant requiring swizzle at program load)
> .... amd would be the same/constant value, regardless of the address
> that the program image was loaded. I used the above convention a lot
> in the early 70s for relocable shared segments.

This reminds me of the way Macintosh OS used to work. (Does it
still do that?) Fortunately I never wrote any low level Mac
code, but you had to be careful when things could be relocated,
and not store addresses across system calls that could move
things around.

In the code above, you can't move things around between the AR
and the BALR. The original mac didn't do preemptive
multitasking, so the user program always knew when things could
move and when they couldn't.

On the other hand, for a system like 680x0 with separate address
and data registers, maybe it isn't so hard.

-- glen

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 09:47:44 -0700

CBFalconer writes:
> This technique has been around for a long time, and is the heart of
> the old CP/M PRL (page relocatable) format. I had it highly
> automated, as exemplified by the program CCINSTAL and batch job
> MAKEDDT.JOB contained in ddtz27.zip. My CCINSTAL code is dated
> 1986, and was not new then. You can examine it all at:
>
>

i had done stuff in the early 70s for relocatable shared segments
.... i.e. the same code running concurrently in the same shared segment
that resided at different virtual addresses in different virtual
address spaces ... random past posts on the subject
http://www.garlic.com/~lynn/subtopic.html#adcon

i was trying to deal with the problem from a structure that was
basically os/360 conventions with relocatable adcons .... a feature of
executable binaries when stored on disk ... and which would be updated
with fixed addresses when the binaries were brought into memory for
execution. the problem was that the embedded fixed addressess would be
unique to specific location of the executable binary. basically i had
to make addresses relative to value that could be calculated on the
fly.

not quite ten years earlier, tss/360 had dealt with the issue by
separating the code and addresses (as data) ... and that the addresses
could be located in private memory area separate from the instruction.

there were other systems from the 60s that also addressed the same
issue.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	CBFalconer
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 16:19:47 GMT

Stephen Pelc wrote:
> (s.subbarayan) wrote:
>
>> Our application sits in a particular location in RAM.The requirement
>> states we have to test entire area of RAM including the location where
>> the code is sitting.(Code in this sense,entire binary image which
>> includes code,data,stack,bss).So we came up with the idea of testing
>> the areas other then where our image sits and then move our image to
>> different location with in RAM which is already tested and then come
>> back to original location.
>
> We've had to write relocatable binaries on systems that do not
> well support position independent code. Our solution is most
> easily implemented using a few lines of assembler for some code
> that calculates the base address at which the code is actually
> running.
>
> We compile/link for two different addresses, and produce a data
> table that holds the addresses of the differences. This table is
> added to the binary. A small piece of start up code applies fixes.
> We usually write this start up code in assembler, and it will
> require the link address.
>
> From our point of view, this technique is particularly attractive
> because it insulates us from many of the details of the final
> executable format if it is not a simple memory image.

This technique has been around for a long time, and is the heart of
the old CP/M PRL (page relocatable) format. I had it highly
automated, as exemplified by the program CCINSTAL and batch job
MAKEDDT.JOB contained in ddtz27.zip. My CCINSTAL code is dated
1986, and was not new then. You can examine it all at:

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson

From:	Anne & Lynn Wheeler
Subject:	Re: Relocating application architecture and compiler support
Date:	Fri, 21 Jan 2005 14:49:09 -0700

so tss/360 had page mapped filesystem as standard and an objective
that a portion of virtual address space could be mapped to the
executable image out in the filesystem ... and that the mapping
process didn't have to track thru arbritrary virtual pages of that
executable image ... swizzling address constant. The system could do
the mapping and start program execution (almost) immedately ...
possibly w/o having to prefetch any of the executable image.

furthermore, if other address spaces were already sharing that
executable image on a r/o basis ... the mapping process would just
setup the segments so the different address spaces utilized the same
virtual pages.

so tss/360 had system wide convention ... assembler, applications,
compilers, etc ... that positioned any address constants that needed
swizzling separately from the executable image. This separate table of
address constants needing swizzling would be prefetched (as part of
mapping the executable file to the virtual address space) and the
necessary swizzle of address constants could be done in a private area
for each specific virtual address space. There then were system wide
conventions on how executable code accessed the table of (swizzled)
address constants .... the executable code image could be exactly the
same for all virtual address spaces ... and in fact, the same exact
executable image could be the same exact virtual pages (using shared
segments) .... but the swizzled address constants would be located in
an (tss/360 system-wide convention) area separate from the executable
program image.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

From:	CBFalconer
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 19:35:52 GMT

Anne & Lynn Wheeler wrote:
>
.... snip ...
>
> not quite ten years earlier, tss/360 had dealt with the issue by
> separating the code and addresses (as data) ... and that the addresses
> could be located in private memory area separate from the instruction.
>
> there were other systems from the 60s that also addressed the same
> issue.

I thought the 360 architecture handled relocation very simply from
the beginning, with base registers.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson

From:	glen herrmannsfeldt
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 13:34:58 -0800

CBFalconer wrote:
(snip)

> I thought the 360 architecture handled relocation very simply from
> the beginning, with base registers.

One of the reasons for base registers is to keep code size down.
In the days of 24 bit address space on 32K byte machines, it
is not necessary to use 24 instruction bits for each address.

Base plus 12 bit displacement uses 16 instruction bits, and
allows for a variety of different addressing modes using one
addressing model.

1) Absolute addressing within the first 4K.

2) With one base register, addressing within a 4K CSECT
often including data and code.

3) Indirect addressing by loading an address into a register
and using that in the instruction.

4) Indexed using the index field of RX instructions, or
computing the address using other instructions.

To do relocation using base registers would require that the OS
know which registers contained addresses. There is no hardware
support for that.

-- glen

From:	Grant Edwards
Subject:	Re: Relocating application architecture and compiler support
Date:	19 Jan 2005 14:21:52 GMT

On 2005-01-19, s.subbarayan wrote:

> Can some one advice me how to create relocatable application in C?

Sure. When you compile your files, tell your compiler you want
it to generate position independant code. When you link your
object modules into an executable image, tell it you want it to
generate position independant code.

> Now whats the way to do this relocation incase my compiler does not
> provide support for creating relocatable image?

None.

--
Grant Edwards grante Yow! Dizzy, are we
at "REAL PEOPLE" or "AMAZING
visi.com ANIMALS"?

From:	Jack Klein
Subject:	Re: Relocating application architecture and compiler support
Date:	Wed, 19 Jan 2005 22:31:09 -0600

On 18 Jan 2005 20:48:49 -0800, s_subbarayan@rediffmail.com
(s.subbarayan) wrote in comp.arch.embedded:

> Embedded gurus,
> Can some one advice me how to create relocatable application in C?We
> are currently working in an application,where we are supposed to test
> the entire RAM area for physical errors in the chip.We have created
> our application for this.The problem we are facing is this:
>
> Our application sits in a particular location in RAM.The requirement
> states we have to test entire area of RAM including the location where
> the code is sitting.(Code in this sense,entire binary image which
> includes code,data,stack,bss).So we came up with the idea of testing
> the areas other then where our image sits and then move our image to
> different location with in RAM which is already tested and then come
> back to original location.
>
> But the problem with this solution is ,we are not aware how this
> relocation can be done?Does this relocation require compiler
> support?In the linker options used in our linker,we have options to
> create relocatable modules.But my colleague says it relocates only the
> code not the data and section areas.When I mean "relocatable module"
> what does it generally mean?
>
> Now whats the way to do this relocation incase my compiler does not
> provide support for creating relocatable image?
>
> Looking farward for all your replys and advanced thanks for the same,
> Regards,
> s.subbarayan.

I never have, and probably never will, write a RAM test in C, or for
that matter one that runs in RAM. Although under some circumstances
you might have to write one that runs from internal RAM on a
microcontroller.

Your executable code exists in some sort of non-volatile memory at
power up, obviously, most likely EPROM or flash. These days there are
two possibilities, as a lot of microcontrollers have routines in
internal ROM to copy code from a serial EEPROM device into internal
RAM to boot from. No code you write ever runs until after it has been
loaded into RAM.

But in the more general case, you have a flash or EPROM connected to
the processor in the address range where start-up code is fetched.
Because that device is narrow or slow or both, you want to copy your
code from the flash or EPROM to RAM for faster execution. If your
memory test code is copied into untested RAM, and the RAM is bad, the
test will never run and your system is going off into never land when
it jumps into the copied code.

The only robust way to do it, to guarantee that your RAM test will
identify bad RAM and not crash or go crazy, is to run your test in
place from the EPROM or flash, and don't use any of the RAM you are
testing for anything, such as stack or variable storage. That's easy
to do in assembly language, but ranges from difficult to impossible in
C.

I have successfully written memory tests this way, for systems with no
internal RAM on the processor/microcontroller, that will correctly run
with no RAM installed on the board, and indicate a RAM failure of
course.

A RAM test that can't run unless at least part of the RAM is already
good is for show only, just window dressing. It's absolutely useless
for high reliability or safety critical systems.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.contrib.andrew.cmu.edu/~ajo/docs/FAQ-acllc.html

From:	CBFalconer
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 06:01:52 GMT

Jack Klein wrote:
>
.... snip ...
>
> A RAM test that can't run unless at least part of the RAM is already
> good is for show only, just window dressing. It's absolutely useless
> for high reliability or safety critical systems.

But you can use a pretty simple test for that preliminary work. I
used to read a location, save in register, complement value, write,
compare reg/memory, complement again (to restore), write, compare
reg/memory. This could test everything except the few instructions
used within that loop.

I put this all in a routine that used only registers, including the
return address. It was called with start/end/return addresses to
test, and returned a condition flag. On failure the address
register indicated the problem location. It also served nicely to
find the top of read/write memory installed.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson

From:	Dave Hansen
Subject:	Re: Relocating application architecture and compiler support
Date:	Thu, 20 Jan 2005 15:28:15 GMT

On Wed, 19 Jan 2005 22:31:09 -0600, Jack Klein
wrote:

[...]
>
>I never have, and probably never will, write a RAM test in C, or for
>that matter one that runs in RAM. Although under some circumstances

I wrote one if Forth once, though it could have been written in C.

Kind of a special circumstance, though. I was testing the RAM on a
prototype video capture board plugged into the PCI bus of a desktop
PC. This was for hardware development/bring up rather than
production/end user use. Somewhat more thorough than the standard
power-up read/write RAM test most embedded system go through -- I was
looking for crossed/stuck address/decode lines more than stuck bits.

Took longer, too. But speed of execution wasn't the driving issue,
speed of implementation was. Forth provided an interactive
environment. C didn't.

But I assumed the PC RAM was fine. ;-)

Regards,

-=Dave
--
Change is inevitable, progress is not.

From:	Niklas Holsti
Subject:	Re: Relocating application architecture and compiler support
Date:	Wed, 19 Jan 2005 21:36:56 +0200

s.subbarayan wrote:
> Embedded gurus,
> Can some one advice me how to create relocatable application in C?We
> are currently working in an application,where we are supposed to test
> the entire RAM area for physical errors in the chip.We have created
> our application for this.The problem we are facing is this:
>
> Our application sits in a particular location in RAM.The requirement
> states we have to test entire area of RAM including the location where
> the code is sitting.(Code in this sense,entire binary image which
> includes code,data,stack,bss).So we came up with the idea of testing
> the areas other then where our image sits and then move our image to
> different location with in RAM which is already tested and then come
> back to original location.

To me that seems like a complex and risky solution. For an alternative
solution, and assuming that your memory-testing function can restore the
original content of each memory location after testing it, how about
having two copies of the test function in the application? Each copy
uses its own (small) area of memory. The first copy tests the memory
(code and data) used by the second copy, and the second copy tests
everything else, including the first copy but not its own memory. In
this way, the application can stay in one place, and memory locations
are always inactive while they are being tested.

If your memory-testing function is complex enough to need a stack, you
may have to use different stack areas for each copy. A simple testing
function could be implemented with only registers and static variables
so it could check the stack area without problems.

I'm assuming that the memory test is done with interrupts disabled and
only one active thread. If the testing has to be done while the
application is running, the solution must be very different.

--
Niklas Holsti
Tidorum Ltd

niklas holsti tidorum fi
. @ .

From:	George Neuner
Subject:	Re: Relocating application architecture and compiler support
Date:	Mon, 24 Jan 2005 02:24:39 -0500

On Wed, 19 Jan 2005 21:36:56 +0200, Niklas Holsti
wrote:

>s.subbarayan wrote:
>> Embedded gurus,
>> Can some one advice me how to create relocatable application in C?We
>> are currently working in an application,where we are supposed to test
>> the entire RAM area for physical errors in the chip.We have created
>> our application for this.The problem we are facing is this:
>>
>> Our application sits in a particular location in RAM.The requir