Why was Intel a no-show on No Execute?

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

In comp.sys.ibm.pc.hardware.chips Yousuf Khan <bbbl67@ezrs.com> wrote:
> Jerry Peters <jerry@example.invalid> wrote:
> >> Each could be upto 32-bits long, meaning 4GB's per segment. How big
> >> did you want them to be?
> >
> > Not unless the code segment overlaps the data segment. Segmented
> > addresses are translated to linear addresses, and if paging is on, the
> > linear address is translated by the mmu to a physical address. The
> > virtual address space is still only 4GB. Segments only subdivide the
> > 4GB address space.
>
>
> True, but we're not talking about a pure segments-only arrangement. We're
> talking about a segmented/paged arrangement. Each segment could could have
> its own separate page entries in the page directory. Meaning that with
> demand-paging only the sections of a segment required to be in memory could
> be in memory while the rest remains marked virtual on disk.
>
> Yousuf Khan
>
No, that still doesn't get you an address space > 4GB.

"Each segment could could have its own separate page entries in the
page directory." What the heck does this mean? A segment doesn't "have"
any entries in the page directory, the linear address which results
from segmentation gets translated to a physical address using the
page directory & page table entries. A linear address is 32 bits,
that's 4GB of address space.

Jerry

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

In comp.sys.ibm.pc.hardware.chips Yousuf Khan <bbbl67@ezrs.com> wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> >> You didn't need long pointers at all. Stack access instructions were
> >> assumed to be coming from the SS segment. Data access instructions
> >> were assumed to be coming from the DS segment (though you could
> >> override with ES, FS, and GS, as well).
> >
> > In C, a "data pointer" might point to an object on the stack. Of
> > course, static analysis can sometimes figure out whether it's
> > pointing to the stack or not, but in most cases it can't know, so you
> > need long
> > pointers everywhere :-(
>
> Well, I'm sure the C compiler implementation would know when it's accessing
> a stack element or a data element, and use the appropriate pointers (i.e.
> stack or data). When are the only times you'd be pointing to an element on
> the stack? It's usually only done when accessing a passed argument. You
> should know which arguments are passed into a function, and which ones are
> local to the function.

In a function, automatic local variables are on the stack. I can pass
the address of one of these to any function which is expecting a
pointer of that type, I could also pass an address of a global or an
address returned by malloc, all in the same function call; think
printf and friends and character string arguments. In fact, with a
literal format string and using gcc defaults, I'd also be passing a
pointer in the text area of the program since gcc defaults to putting
ro strings in the code area.

Jerry


---snip-----

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Thu, 3 Jun 2004, Yousuf Khan wrote:

> Robert Wessel <robertwessel2@yahoo.com> wrote:
> > No. It's quite common to pass a pointer to a piece of automatic (on
> > stack) storage to a function accepting pointers to that type of data.
> > Or a pointer to global, or allocated storage. Unless you pass some
> > additional data (in x86, a selector, for example), the called routine
> > has no way to tell which segment to look in. Consider:
>
> Well, as I said in another message, if stack and data need to share the same
> segment, then you can share the same segment for them. You still don't have
> to combine them into the same segment as the code segment.

What on Earth has that got to do with anything? You still need long
pointers -- if your compiler is heroic, it can optimize some of the
pointer operations into short pointers (i.e. 4-byte pointers).

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Peter "Firefly" Lund wrote:
> On Thu, 3 Jun 2004, Yousuf Khan wrote:
>
> > Well, as I said in another message, if stack and data need to share
> > the same segment, then you can share the same segment for them. You
> > still don't have to combine them into the same segment as the code
> > segment.
>
> What on Earth has that got to do with anything? You still need long
> pointers

What for? If 'DS == SS' then a pointer dereferenced through either
segment will refer to the same memory location. All that's necessary is
to restrict the size of the code segment so that it doesn't include
memory used for the heap or stack.

--
Wishing you good fortune,
--Robin Kay-- (komadori)

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Thu, 3 Jun 2004, Jerry Peters wrote:

> No, that still doesn't get you an address space > 4GB.

Pointer swizzling.

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

You then can only have only one code segment and only one data segment,
which at the same time should be stack segment.

You then have to reserve a whole area for code only, and load all code
segments of all dynamic libraries there, and reserve the rest for data. You
won't be able to arbitrarily mix data and code areas - it would require
'far' pointers, which are a pain.

"Yousuf Khan" <bbbl67@ezrs.com> wrote in message
news:ilJvc.9368$bVw1.4708@news01.bloor.is.net.cable.rogers.com...
> Robert Wessel <robertwessel2@yahoo.com> wrote:
> > No. It's quite common to pass a pointer to a piece of automatic (on
> > stack) storage to a function accepting pointers to that type of data.
> > Or a pointer to global, or allocated storage. Unless you pass some
> > additional data (in x86, a selector, for example), the called routine
> > has no way to tell which segment to look in. Consider:
>
> Well, as I said in another message, if stack and data need to share the
same
> segment, then you can share the same segment for them. You still don't
have
> to combine them into the same segment as the code segment.
>
> Yousuf Khan
>
>

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Jerry Peters <jerry@example.invalid> wrote:
> No, that still doesn't get you an address space > 4GB.
>
> "Each segment could could have its own separate page entries in the
> page directory." What the heck does this mean? A segment doesn't
> "have" any entries in the page directory, the linear address which
> results from segmentation gets translated to a physical address
> using the page directory & page table entries. A linear address is
> 32 bits, that's 4GB of address space.

Well, when you're using a task gate to switch between programs, you can set
each task to have its own unique page table base address (loaded into CR3).
Each task could have its own page table. You can map all programs to have
their own unique pages, including unique virtual pages.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Jerry Peters <jerry@example.invalid> wrote:
> No, that still doesn't get you an address space > 4GB.
>
> "Each segment could could have its own separate page entries in the
> page directory." What the heck does this mean? A segment doesn't
> "have" any entries in the page directory, the linear address which
> results from segmentation gets translated to a physical address
> using the page directory & page table entries. A linear address is
> 32 bits, that's 4GB of address space.

Well, when you're using a task gate to switch between programs, you can set
each task to have its own unique page table base address (loaded into CR3).
Each task could have its own page table. You can map all programs to have
their own unique pages, including unique virtual pages.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Jerry Peters <jerry@example.invalid> wrote:
> No, that still doesn't get you an address space > 4GB.
>
> "Each segment could could have its own separate page entries in the
> page directory." What the heck does this mean? A segment doesn't
> "have" any entries in the page directory, the linear address which
> results from segmentation gets translated to a physical address
> using the page directory & page table entries. A linear address is
> 32 bits, that's 4GB of address space.

Well, when you're using a task gate to switch between programs, you can set
each task to have its own unique page table base address (loaded into CR3).
Each task could have its own page table. You can map all programs to have
their own unique pages, including unique virtual pages.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Alexander Grigoriev <alegr@earthlink.net> wrote:
> You then can only have only one code segment and only one data
> segment, which at the same time should be stack segment.
>
> You then have to reserve a whole area for code only, and load all code
> segments of all dynamic libraries there, and reserve the rest for
> data. You won't be able to arbitrarily mix data and code areas - it
> would require 'far' pointers, which are a pain.

You can use the page tables to map the data pages totally separately from
the code pages. In a case like that you can start your data access right
from address 0 and finish right at the 4GB address mark. There will be no
danger of it overwriting the code, because the code will have totally
different pages. The code itself could go from 0 to 4GB without ever
trampling over the data segments either. This where the paging magic comes
in.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

"Yousuf Khan" <bbbl67@ezrs.com> wrote in message
news:vA%vc.1889$0h.537@news04.bloor.is.net.cable.rogers.com...
> Alexander Grigoriev <alegr@earthlink.net> wrote:
> > You then can only have only one code segment and only one data
> > segment, which at the same time should be stack segment.
> >
> > You then have to reserve a whole area for code only, and load all code
> > segments of all dynamic libraries there, and reserve the rest for
> > data. You won't be able to arbitrarily mix data and code areas - it
> > would require 'far' pointers, which are a pain.
>
> You can use the page tables to map the data pages totally separately from
> the code pages. In a case like that you can start your data access right
> from address 0 and finish right at the 4GB address mark. There will be no
> danger of it overwriting the code, because the code will have totally
> different pages. The code itself could go from 0 to 4GB without ever
> trampling over the data segments either. This where the paging magic comes
> in.

While the physical address space can be 36-bit, isn't the linear address
space still limited to 32-bit? If that's correct, all segments must have
the same base (zero) if they're to have a 4GB limit.

S

--
Stephen Sprunk "Stupid people surround themselves with smart
CCIE #3723 people. Smart people surround themselves with
K5SSS smart people who disagree with them." --Aaron Sorkin

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Fri, 4 Jun 2004, Robin KAY wrote:

> Peter "Firefly" Lund wrote:
> > On Thu, 3 Jun 2004, Yousuf Khan wrote:
> >
> > > Well, as I said in another message, if stack and data need to share
> > > the same segment, then you can share the same segment for them. You
> > > still don't have to combine them into the same segment as the code
> > > segment.
> >
> > What on Earth has that got to do with anything? You still need long
> > pointers
>
> What for? If 'DS == SS' then a pointer dereferenced through either
> segment will refer to the same memory location. All that's necessary is
> to restrict the size of the code segment so that it doesn't include
> memory used for the heap or stack.

I think you misunderstood Yousuf.

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Stephen Sprunk <stephen@sprunk.org> wrote:
> While the physical address space can be 36-bit, isn't the linear
> address space still limited to 32-bit? If that's correct, all
> segments must have the same base (zero) if they're to have a 4GB
> limit.

Yes, but they can be different 4GB regions because they will have different
page mappings.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Peter "Firefly" Lund <firefly@diku.dk> wrote:
>> What for? If 'DS == SS' then a pointer dereferenced through either
>> segment will refer to the same memory location. All that's necessary
>> is to restrict the size of the code segment so that it doesn't
>> include memory used for the heap or stack.
>
> I think you misunderstood Yousuf.

Nope, he understood it right.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

"Yousuf Khan" <bbbl67@ezrs.com> wrote in message
news:Vl3wc.8$sAv.4@news04.bloor.is.net.cable.rogers.com...
> Stephen Sprunk <stephen@sprunk.org> wrote:
> > While the physical address space can be 36-bit, isn't the linear
> > address space still limited to 32-bit? If that's correct, all
> > segments must have the same base (zero) if they're to have a 4GB
> > limit.
>
> Yes, but they can be different 4GB regions because they will have
different
> page mappings.

How? LDT/GDT are used to map virtual addresses to linear addresses, and
page tables are used to map linear addresses to physical addresses. If the
linear address space is 32-bit, how can a single process address more than
4GB in unique virtual address space? My understanding is that systems over
4GB use different page directories for each process so that the linear
address space can be reused, but I don't see a similar scheme using
selectors within a single process.

S

--
Stephen Sprunk "Stupid people surround themselves with smart
CCIE #3723 people. Smart people surround themselves with
K5SSS smart people who disagree with them." --Aaron Sorkin

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

In comp.sys.ibm.pc.hardware.chips Yousuf Khan <bbbl67@ezrs.com> wrote:
> Jerry Peters <jerry@example.invalid> wrote:
> > No, that still doesn't get you an address space > 4GB.
> >
> > "Each segment could could have its own separate page entries in the
> > page directory." What the heck does this mean? A segment doesn't
> > "have" any entries in the page directory, the linear address which
> > results from segmentation gets translated to a physical address
> > using the page directory & page table entries. A linear address is
> > 32 bits, that's 4GB of address space.
>
> Well, when you're using a task gate to switch between programs, you can set
> each task to have its own unique page table base address (loaded into CR3).
> Each task could have its own page table. You can map all programs to have
> their own unique pages, including unique virtual pages.
>
> Yousuf Khan
??? You're confused. That's how Linux works now, each process has its
own page tables and hence address space. Oh, except Linux doesn't use
a task gate, since 1) it's slow, & 2) limits you to something like
4000 total processes. This still doesn't get you a _single_ AS that's
greater then 4GB. AFAIK x86 has no facilities like IBM's S3X0 has to
have access to multiple address spaces at once.

You _really_ do need to do some research on OS design and
implementation issues.

Jerry

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Jerry,

Windows works the same way. Each process has its owns
set of page tables. That is how Windows multitasking
works. Part of a process switch is to switch the current set of
page tables. That is how the same virtual address in each process
accesses different physical address.


"Jerry Peters" <jerry@example.invalid> wrote in message
news😛X6wc.5076$Gx4.1972@bgtnsc04-news.ops.worldnet.att.net...
> In comp.sys.ibm.pc.hardware.chips Yousuf Khan <bbbl67@ezrs.com> wrote:
> > Jerry Peters <jerry@example.invalid> wrote:
> > > No, that still doesn't get you an address space > 4GB.
> > >
> > > "Each segment could could have its own separate page entries in the
> > > page directory." What the heck does this mean? A segment doesn't
> > > "have" any entries in the page directory, the linear address which
> > > results from segmentation gets translated to a physical address
> > > using the page directory & page table entries. A linear address is
> > > 32 bits, that's 4GB of address space.
> >
> > Well, when you're using a task gate to switch between programs, you can
set
> > each task to have its own unique page table base address (loaded into
CR3).
> > Each task could have its own page table. You can map all programs to
have
> > their own unique pages, including unique virtual pages.
> >
> > Yousuf Khan
> ??? You're confused. That's how Linux works now, each process has its
> own page tables and hence address space. Oh, except Linux doesn't use
> a task gate, since 1) it's slow, & 2) limits you to something like
> 4000 total processes. This still doesn't get you a _single_ AS that's
> greater then 4GB. AFAIK x86 has no facilities like IBM's S3X0 has to
> have access to multiple address spaces at once.
>
> You _really_ do need to do some research on OS design and
> implementation issues.
>
> Jerry

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

In comp.sys.ibm.pc.hardware.chips "Peter \"Firefly\" Lund" <firefly@diku.dk> wrote:
> On Thu, 3 Jun 2004, Jerry Peters wrote:
>
> > No, that still doesn't get you an address space > 4GB.
>
> Pointer swizzling.
>
> -Peter

Yeah, just what the world needs, a swapping system for segments. Can
you say _slow_? Given the context of this discussion, about using NX
segments, can you imagine the thrashing with disjoint 4GB code & data
segments? That's what Yousef's proposing you realize.

Jerry

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Fri, 4 Jun 2004, Yousuf Khan wrote:

> Peter "Firefly" Lund <firefly@diku.dk> wrote:
> >> What for? If 'DS == SS' then a pointer dereferenced through either
> >> segment will refer to the same memory location. All that's necessary
> >> is to restrict the size of the code segment so that it doesn't
> >> include memory used for the heap or stack.
> >
> > I think you misunderstood Yousuf.
>
> Nope, he understood it right.

Sorry, then you misunderstood something.

My apologies to Robin Kay.

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Fri, 4 Jun 2004, Stephen Sprunk wrote:

> > Yes, but they can be different 4GB regions because they will have
> different
> > page mappings.
>
> How? LDT/GDT are used to map virtual addresses to linear addresses, and
> page tables are used to map linear addresses to physical addresses. If the
> linear address space is 32-bit, how can a single process address more than
> 4GB in unique virtual address space? My understanding is that systems over

It can't, really. But...

> 4GB use different page directories for each process so that the linear
> address space can be reused, but I don't see a similar scheme using
> selectors within a single process.

....if you have long pointers (with selectors), several of them, in fact,
then you can mark some of the corresponding segments as not present.
When the program tries to access it, the operating system can kick and
change the page tables (and mark the faulting segment present). You have
in effect taken the bank switching schemes from the late eighties' home
computers into the brave new modern world. Or something.

It's called pointer swizzling.

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

On Fri, 4 Jun 2004, Jerry Peters wrote:

> In comp.sys.ibm.pc.hardware.chips "Peter \"Firefly\" Lund" <firefly@diku.dk> wrote:
> > On Thu, 3 Jun 2004, Jerry Peters wrote:
> >
> > > No, that still doesn't get you an address space > 4GB.
> >
> > Pointer swizzling.
> >
> > -Peter
>
> Yeah, just what the world needs, a swapping system for segments. Can
> you say _slow_? Given the context of this discussion, about using NX
> segments, can you imagine the thrashing with disjoint 4GB code & data
> segments? That's what Yousef's proposing you realize.

/I/ know that 😉

-Peter

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Jerry Peters <jerry@example.invalid> wrote:
>> Pointer swizzling.
>>
>> -Peter
>
> Yeah, just what the world needs, a swapping system for segments. Can
> you say _slow_? Given the context of this discussion, about using NX
> segments, can you imagine the thrashing with disjoint 4GB code & data
> segments? That's what Yousef's proposing you realize.

Not necessary, the pages will take care of the pointers.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Stephen Sprunk <stephen@sprunk.org> wrote:
>> Yes, but they can be different 4GB regions because they will have
>> different page mappings.
>
> How? LDT/GDT are used to map virtual addresses to linear addresses,
> and page tables are used to map linear addresses to physical
> addresses. If the linear address space is 32-bit, how can a single
> process address more than 4GB in unique virtual address space? My
> understanding is that systems over 4GB use different page directories
> for each process so that the linear address space can be reused, but
> I don't see a similar scheme using selectors within a single process.

It's been awhile since this stuff has swirled around in my head, I remember
the last time I really looked at it serious was back in the 386 days. Each
of the segments would have to have different page directories. And yes, I
guess if they have different page directories, then that would mean that the
segments need to occupy different linear addresses. However, the relative
offsets within each segment would still be from 0 to some number.

Yousuf Khan

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

"Yousuf Khan" <bbbl67@ezrs.com> writes:

>It's been awhile since this stuff has swirled around in my head, I remember
>the last time I really looked at it serious was back in the 386 days. Each
>of the segments would have to have different page directories.

Segments don't have page directories now, and segments didn't have page
directories back on the 386. The page directory base register is a single
global processor register, reloaded either explicitly (load to %cr3, IIRC),
or implicitly through a task gate control transfer between processes
(which is slow, and probably never used in modern OSses).

The only way I know of which could in theory give a single process
more than 32 bits of accessible address space, is the constant making
present / nonpresent of segments, with the kernel fault handler for
nonpresent segments fiddling with the page directory. However, as you
can imagine, that means a user/kernel/user switch whenever a nonpresent
segment is used, accompanied by TLB invalidation stuff on page directory
reloading.

Is anybody aware of a general purpose OS that did/does such segment twiddling?

best regards
Patrick

 

[Guest]

Archived from groups: comp.arch,comp.sys.ibm.pc.hardware.chips,comp.sys.intel (More info?)

Patrick Schaaf <mailer-daemon@bof.de> wrote:
> "Yousuf Khan" <bbbl67@ezrs.com> writes:
>
>> It's been awhile since this stuff has swirled around in my head, I
>> remember the last time I really looked at it serious was back in the
>> 386 days. Each of the segments would have to have different page
>> directories.
>
> Segments don't have page directories now, and segments didn't have
> page directories back on the 386.

I know that, I didn't mean they were literally linked, I meant it had to be
linked through a software setup inside the kernel.

> The only way I know of which could in theory give a single process
> more than 32 bits of accessible address space, is the constant making
> present / nonpresent of segments, with the kernel fault handler for
> nonpresent segments fiddling with the page directory. However, as you
> can imagine, that means a user/kernel/user switch whenever a
> nonpresent segment is used, accompanied by TLB invalidation stuff on
> page directory reloading.

Which is the method I was actually thinking of. However, as you said it is a
slow process. It would be a less slow process if the data and code segments
occupied different locations in linear memory, therefore they could share
the same page table directories without requiring special OS-based software
page table switching techniques.

> Is anybody aware of a general purpose OS that did/does such segment
> twiddling?

I was thinking the pre-1.0 Linux kernels did it, but back then processes
were limited to 16MB of memory each in Linux, so therefore they could've all
fit into a single page directory, so it wasn't the ideal example.

Yousuf Khan